serverless container disk storage size vs network volume
When I add a Serverless endpoint, it defaults to 5GB container disk. I tried to change it to a crazy high number like 50000GB, and it seems to be ok with it??
I'm confused, is this disk storage physically attached to the GPU machine? there a limit of this storage size? Does it cost any extra money? My ComfyUI docker image needs to download many different models, which can be hundreds of GB in total, what would happen if it exceeds the storage limit?
If I choose to attach a network volume, does it mean my docker image (contains many different models) will be deploy to store in the network volume? And the deployed volume need to communicate with the GPU via network requests?? So it will be slower?
If I have 4 GPU workers running in parallel, can they share the same network volume?
6 Replies
If I have 4 GPU workers running in parallel, can they share the same network volume?
they can, as long they're in the same region ( secure cloud / serverless )
If I choose to attach a network volume, does it mean my docker image (contains many different models) will be deploy to store in the network volume? And the deployed volume need to communicate with the GPU via network requests?? So it will be slower?
Another way is: you can store just the models in some path in the network volume, that way it will be faster. and i think network volumes are meant to be for storing models. the docker image is from the templates only.
Also network volume is connected using ethernet cables if not wrong, they only communicate with the same region or same datacenter i guess. so they may not be in the same gpu machine
I'm confused, is this disk storage physically attached to the GPU machine? there a limit of this storage size? Does it cost any extra money? My ComfyUI docker image needs to download many different models, which can be hundreds of GB in total, what would happen if it exceeds the storage limit?
Of course there is some physical storage size limits from the machines🤣 , but im not sure if they are on the same machine. Container disk & persistent volume disk on the pod is connected in the same servers i guess.
Well, if it exceeds the storage limit (it cant) your download will be canceled ( since it runs out of space on the disk ) there is some software limits set based on the storage you have created, but no worries you can up the storage size anytime you want!
Hope these help, sorry for not answering them in order haha
Another way is: you can store just the models in some path in the network volume, that way it will be faster. and i think network volumes are meant to be for storing models. the docker image is from the templates only.ohh this makes sense!! Good idea to only store models. I do see people storing the entire a111 in network volume and use it from there: https://github.com/ashleykleynhans/runpod-worker-a1111
GitHub
GitHub - ashleykleynhans/runpod-worker-a1111: RunPod Serverless Wor...
RunPod Serverless Worker for the Automatic1111 Stable Diffusion API - ashleykleynhans/runpod-worker-a1111
Well, if it exceeds the storage limit (it cant)im curious what's the storage limit look like, I see some pod has around 40GB disk size. in serverless machine I assume its about the same? that means i have to store my models in network volume, 40G is not enough for many models
really?
oH ye ya correct, because some of the files that needed to be saved are in that directory too
You can adjust the disk size, it will be that amount that you rent
you actually pay for them too
I see! Thanks. But I assume it couldn’t be too large? Curious about the limit. If I were to guess, I guess it would be under 100gb
Yeah maybe around that
Some pods have bigger size too
try them out, some may haave like 1tb