Huge sudden delay times in serverless
I'm using a webui forge serverless template for my endpoint, with a network volume attached and sometimes the results are very inconsistent. For example in the last two results you can see I use the same worker but one has a delay time of 3s and another 80.39s, the second request was submitted 4-5 seconds later after the first request, so there was no long time gap either.
I know the forge/automatic1111 templates usually take time to load but all this time up until this monday/tuesday or so it only took about 10-20 second delay time, but now I'm having 80-90 second delays. Didn't make any change in my code either. Anyone know the reason for this?...
Testing Async Handler Locally
Hi,
I am trying to test the async handler locally. I am following the documentation very closely (attached is an example of something I tried). However, I cannot seem to get the /status/{job_id} endpoint to return anything while the job is in progress (I expected it to return IN_PROGRESS and perhaps the values that have thus far been yielded) -- that is, I recieve no response. I am testing locally by running the handler with the --rp_serve_api flag.
Is there something I am doing wrong? Why can't I see the status of a job in progress?
...
Why too long delay time even if I have active worker ?
I have set the active worker to 1.
I am manually testing the response delay. I submit the next task only after the previous task is completed, so there is no waiting time.
However, many times, the delay time is still very long, sometimes even reaching more than 4 seconds.
Why is this?
In my code, the model has been loaded before runpod.serverless.start({"handler": run})...
