gego144
Explore posts from servers
RRunPod
•Created by gego144 on 12/12/2024 in #⚡｜serverless
Why is my serverless endpoint requests waiting in queue when theres free workers?
This has been happening,when two people try to make a request at the same time, the second users request will wait in queue until the first request is completed instead of trying to use another worker. I have 4 workers avaliable on my endpoint so thats not the issue. I set the queue delay to 1 second because thats the lowest possible but it doesn't do anything. Is the serverless endpoint suppose to work in production?
6 replies