Keeping idle workers alive even without any requests.
Hey everyone, does anyone have a clear understanding of how idle timeout works on RunPod?
It seems like billing is based on max workers by default.
For instance on this deployment I set 5 max workers, 0 active workers, and an idle timeout of 5 seconds, but even with no requests, I still see 3 idle workers. Is this expected behavior, or is something off?
4 Replies
@kapa.ai
It seems like billing is based on max workers by default.
For instance on this deployment I set 5 max workers, 0 active workers, and an idle timeout of 5 seconds, but even with no requests, I still see 3 idle workers. Is this expected behavior, or is something off?
I want these workers to be turned off after an idle timeout.
Do i understand correctly that if i have 4 idle workers displayed i'm actually paying for 4 workers?
i've just submitted a request to support. Attaching this screenshot as well.
Would appreciate help
It's just an unfortunate feature naming conflict.
Idle worker in the context of endpoint means "initialized worker, without a currently active job, ready to accept one". Idle workers are not billed
Idle timeout in the context of job execution means "how long after a job finishes is the worker kept running, with the ability to instantly execute another request".
So yes, having idle workers is an expected and desired state because it means your endpoint has free workers for incoming requests. On the other hand, setting idle timeout to 600 seconds will make your workers run very long for full price. Either set it to something reasonable like 1-5 seconds or consider using active workers which are discounted, if you can't afford to have cold starts or big delay times.yeah, i set 5 sec, this screenshot is from dev endpoint.
Thank you!