Network volume vs baking in model into docker
I want to run a serverless worker that can get called anywhere from once per hour to 300-400/hour.
I want to optimize for cold starts when the occasional request comes in.
it runs SDXL, a checkpoint, a few controlnets, etc. About 15-20GB in total.
Am I better off using a network volume or putting the images directly into the docker image?
For the initial pod setup on the first request, does it download the entire docker image each time? And thats gonna be slower than reading the network volume?
I saw this earlier: https://www.answeroverflow.com/m/1194344244563943604
[RUNPOD] Minimize Worker Load Time (Serverless) - RunPod
Hey fellow developers,
I'm currently facing a challenge with worker load time in my setup. I'm using a network volume for models, which is working well. However, I'm struggling with Dockerfile re-installing Python dependencies, taking around 70 seconds.
API request handling is smooth, clocking in at 15 seconds, but if the worker goes inactive,...
2 Replies