Serverless Always IN_QUEUE?
Hi, I am pretty new to using Serverless in Runpod, and upon setting up my API and including my docker image with a Python script for making inferences (receiving and returning JSON), all the requests goes in and gets stuck at IN_QUEUE all the time. Testing my inference script locally does not cause any issues, is this an issue with the configuration?
2 Replies
making sure your starting the serverless, usually this occurs when workers are not setup properly and they never pickup a job
The
serverless.Start(....
call in your handler
Or what about your dockerfile how does it calls the py handler file