Serverless vllm running but still downloading?
Title says it all and this shouldn't happen
5 Replies
still
anyways, how to replicate: create a vllm endpoint, then create a network storage, then attach ONLY after its created and all ready
Then send requests when 1 worker is ready in the latest workers, to openAI endpoint
@yhlong00000
@nerdylive
Escalated To Zendesk
The thread has been escalated to Zendesk!
i'll try to
Hi there. This is not optimized behavior, but it is expected behavior., Since when you switch to using a network volume, all the running containers need to be removed, and need to be changed to be from the same data center the network volume is in
I'll speak to the team about optimizing this behavior, but this is an edge case do we likely will take some time to do this.
Nonetheless, the downloading new data occurs because we need to change all the Workers to be only from the data center connected to the network volume.
Yup that should work, Keep it in the same data centre as uh the network volume
would you just be able to always use the network volume hre?