serverless health
https://api.runpod.ai/v2/adaejhk*****/health
The health endpoint while cold starting, never indicate the state of initializing?
It just goes from idle/ready to running and back.
Is there a way to indicate the Serverless Endpoint is warming up for the application indication?
6 Replies
will look into this, init state really only should occur for new endpoints, after that we try to avoid production outage by doing rolling releases
Initializing only refers to the Endpoint when it´s being initially created? Not to the state of the workers? Is there a way to see when the docker container is being downloaded, or the GPU Model is being loaded, that we can Update the state?
{
"jobs": {
"completed": 367,
"failed": 6,
"inProgress": 0,
"inQueue": 0,
"retried": 0
},
"workers": {
"idle": 1,
"initializing": 0,
"ready": 1,
"running": 0,
"throttled": 0,
"unhealthy": 0
}
}
No, maybe it's for workers, but after it is done ( idle) and you deploy new releases it will use rolling release which uses some of your ready (idle) workers and update them with the new image you just deploy, then gradually update the others too when the first ones are back ready
And for the state updates yes you can do it, but not from Runpod's system.. More likely it is from your program or apication that you use to run the model so it's customizable from your code , you can update status from the worker code
You can click each worker to see the image download status too
init means image is being downloaded, ready and idle are the samething, idle is the new naming, init is status for workers, if all workers are in init state, we show the endpoint is initializing
I don´t see this alignment betweeen the status of the endpoint and what seems to be going on, while the host in in Running state I see the image is being downloaded.
that should not happen unless your updating container images and using same tag, thats the only edge case we know of so far