Do you get charged whilst your request is waiting on throttled workers?
I did everything I can to reduce cold start times on my end. I managed to install everything onto a Docker container (and avoid using Network Volume), and I'm using the official runpod A11111 worker with minor modifications.
Unfortunately the cold start times are still random. I noticed that despite setting 3 Max workers to an endpoint, all 5 workers would get the "Throttled" status, which I'm guessing is the reason why the cold start times are so random.
If I'm getting a very high request time due to Throttled workers, in this case, one basic request took 170 seconds - am I getting charged for the entire 170 seconds?
5 Replies
You should only be getting charged if your workers are Active (green on the dashboard). If they're throttled then someone else is using the GPU and you shouldn't be getting charged.
Is there a way to see what portion of the Delay Time was charged and what part was not?
Nope, not possible. You are only charged for the cold start time, not the part while the request is sitting in the queue.
you can check your dashboard, charged for cold start + execution time, delay time is there for extra telemetry but not charged
Thank you guys for clearing that up