morrow
RRunPod
•Created by morrow on 1/2/2025 in #⚡|serverless
Best way to cache models with serverless ?
Hello,
I'm using serverless endpoint to do image generation with flux dev. The model is 22gb which is quite long to download, especially since some workers seem to be faster than some others.
I've been using a network volume as a cache which greatly improve start up time. However, doing this lock me in a particular region which I believe make some GPUs like the A100 very rarely available.
Is there a way to have a global huggingface cache with serverless endpoint ? (like with pods)
Thanks
6 replies
RRunPod
•Created by morrow on 12/8/2024 in #⚡|serverless
Serverless github endpoint stuck at uploading phase
![No description](https://cdn.answeroverflow.com/1315429347410051092/image.png)
11 replies