Problems with serverless trying to use instances that are not initialized
We are having problems with serverless endpoints reguarly. For some reason it tries to route request to instances that are not initialized in reality but RunPod seems to think they are initialized. This causes RunPod to show the entire 10 min initialization as "Running" even though it's not.
5 Replies
It also keeps redownloading these images on containers that are supposedly "running" quite often so it becomes pretty unusable
this happens with multiple different endpoints
Here is another one, it wrongly categorizes the container that is downloading the image as "idle"
Yes the running flag sometimes appears before the actual launch of the handler function. It should be linked to the inference script, with some kind of test inference before routing the system to that pod.
Has this endpoint not been used for a while, or did you just create it and notice the UI showing green while it’s still downloading the image?