Intermittent timeouts on requests
I have a custom docker image serverless endpoint. I am sending a payload with the python package endpoint.run_sync(payload, timeout=60). I currently have 0 active workers.
I can typically send the first request. After completion, if I send a following request before that worker times out, it will often timeout without ever logging that the main function started. Basically, it seems like the message never gets in the RunPod Queue. What could be causing this behavior? How can I avoid it (or debug it)?
Logs are attached - this case is 2 successful requests, then a third request just times out - it seems like the request never gets to the queue (no logs)....
Logs are attached - this case is 2 successful requests, then a third request just times out - it seems like the request never gets to the queue (no logs)....
GPU Availability Issue on RunPod – Need Assistance
Hi everyone,
I’m currently facing an issue with GPU availability for my ComfyUI endpoint (id: kw9mnv7sw8wecj) on RunPod. When trying to configure the worker, all GPU options show as “Unavailable”, including 16GB, 24GB, 48GB, and 80GB configurations (as shown in the attached screenshot).
This is significantly impacting my workflow and the ability to deliver results to my clients since I rely on timely image generation....
Not using cached worker
I've been running into this problem for several days now. I have a endpoint that runs a forge webui worker with a network volume attached. And as you know forge takes some time to start and only then generates the image. So generally when I send a request to a worker it takes some delay for the start process then generates images.
But recently I've run into an issue where there is already a worker running with webui forge started and ready to accept requests but when I submit a new request it completely starts a new worker, which results in huge delay times. My question is, why isn't it using the already available worker which has forge loaded?
And no, the requests weren't submitted one after the other so there is no reason to start a new worker...
