Can I emulate hitting serverless endpoints locally?
So far I've been testing my runpod serverless locally by executing the python handler
but is there any way to emulate hitting the serverless endpoint locally?
14 Replies
ok so I found out that we can serve it using
--rp_serve_api but is there any way to make it mimic serverless workers fully? Key differences that I noticed are:
- webhooks got their response only after I hit the status endpoint
- local webhook response differ from the webhook response of the actual runpod serverless workers, local response consist of only the output field instead of the full json response schema
is there any way I can do to make them as close as how the actual workers behave? or at least on those 2 pointsNo, unless you modify the Local part of the SDK yourself by forking it. Ive been considering to improve on the local testing, but I gave up on it, and just created a way for me to test on GPU Pod which is the closest env Ill get to, and you can probably add some manual emulations yourself like sending out your own webhooks all that.
https://github.com/justinwlin/Runpod-OpenLLM-Pod-and-Serverless/blob/main/handler.py
My repo I use to test from GPU pod > to deploying on serverless
ah I see thanks! actually I just need to ensure that the webhook response in production and staging are both same, because when we run it locally using that rp serve thing, the webhook response are totally different
@gpu poor We have a CLI tool that can help iterate over your development process quicker.
It has the ability to sync your changes over a Pod - the only thing you'll need to do, is update the Endpoint you're interacting with.
https://docs.runpod.io/cli/install-runpodctl
thanks @PatrickR! but what I need the most is to match this webhook response from local runpod development server and the actual runpod serverless, for example a runpod serverless endpoint webhook would look like this:
but a webhook from the local development server provided would look like this:
that's my main problem rn because I need to use the
input field also while parsing the webhook responseYep, the dev environment creates a Fast API server for you on port
8080 where you can pass in your webhook.
Makes for pretty quick development and you can test your code.@gpu poor I can have the webhook match closer, did not realize we were returning input as part of it.
@Merrell ah I see, thanks! It'd be great to have both the local dev and prod to return a similar schema, it'd helps our local staging a lot
The are apparently 2 versions of the webhook, the v2 webhook doesn't return the input, so if you use that, then dev and prod should match.
@ashleyk ah, didn't know about that. how can I use the v1 instead?
Don't think local testing has the v1 format, sounds like it has the v2 format webhook.
is that the same version that is denoted in my runpod url? i.e.
https://api.runpod.ai/v2/asdfasdf or is it something else