jeffcrouse
RRunPod
•Created by jeffcrouse on 10/2/2024 in #⚡｜serverless
where should I put my 30GB of models?
ok, thanks
15 replies
RRunPod
•Created by jeffcrouse on 10/2/2024 in #⚡｜serverless
where should I put my 30GB of models?
@yhlong00000 Is there an easy way to view the current GPU availability at different data centers so that I can make a more informed choice about which data center I choose for my network volume?
15 replies
RRunPod
•Created by jeffcrouse on 10/2/2024 in #⚡｜serverless
where should I put my 30GB of models?
However, in the "Serverless > New Endpoint" dialog, as soon as I select a network volume, I can only choose the 80 GB GPU PRO worker configuration (the rest are "unavailable"), and only with Max Workers = 1. Is this a known limitation of attaching a network volume?
15 replies
RRunPod
•Created by jeffcrouse on 10/2/2024 in #⚡｜serverless
where should I put my 30GB of models?
I spent all morning converting and testing the approach where I create a network storage volume, populate it with my models, and then attach it to a serverless endpoint, and I can report that it is plenty fast for me.
15 replies
RRunPod
•Created by jeffcrouse on 10/2/2024 in #⚡｜serverless
where should I put my 30GB of models?
After some experimenting, I can answer one of my own questions. One that is probably obvious if one is already familliar with Docker... It's easy to mount a directory from your local file system inside a Docker container, so this is how I can test locally. In fact, this is already set up in runpod-worker-comfy -- I just had to copy my models into data/runpod-volume and this is mounted via the docker-compose.yaml file
15 replies
RRunPod
•Created by jeffcrouse on 10/2/2024 in #⚡｜serverless
where should I put my 30GB of models?
ah, gotcha
15 replies
RRunPod
•Created by jeffcrouse on 10/2/2024 in #⚡｜serverless
where should I put my 30GB of models?
The fact that the Serverless "New Endpoint" form doesn't specifically mention DockerHub seems to suggest that it might be agnostic?
15 replies
RRunPod
•Created by jeffcrouse on 10/2/2024 in #⚡｜serverless
where should I put my 30GB of models?
Sorry, I don't know about any other container registries. Are there advantages to the other ones? I was worried that Docker Hub would reject mine because I had read that they have 10GB/layer size limit, but I was pleased to find that it pushed successfully.
15 replies