Quick question about what "Extra workers" are
I hit a serverless endpoint a bunch of times in succession, and it spun up two other workers, which is GREAT!
But they spun up the "Extra Workers" instead of the "Latest Workers".
That prompted me to be confused what the "Extra Workers" are for even.
FYI, I provisioned 3 workers, and it gave me 3 "Latest Workers", and 2 "Extra workers".
Pic attached!
4 Replies
The extra workers are there to help with throttling and not actually supposed to be used, the latest workers are supposed to be used, not the extra workers so sounds like you have found some kind of bug if extra workers are being used instead of latest workers.
ok either that.
or my latest workers got cycled out so quickly and i just don't see the same logs from the previous instance of them.
.-.
i swear i checked immediately tho
You can only really see the logs when they are active, otherwise go to the logs tab for the endpoint.
Just ignore extra workers 🙂 as digigoblin said, sometimes the pool of gpus can be eaten up by a big user so they allocate u extra gpus as a sort of additional gpu cycling measures
i also find the live logging bit weird, but refreshing when they are active, / checking the logging tag after the job is done is usually there
i sometimes find the logs when the job is running lags a bit, but overall im ok with it since they do end up showing in the aggregated log tabs
Just to share:
https://discord.com/channels/912829806415085598/1247594927831908504
Usually for debugging my serverless endpoint my stuff is usually built using this repo as a base, making it easy to just call my handler.py function in a gpu pod and so i can test my handler.py before redeploying to serverless if any issues show up in logs