Serverless container storage
I made a script that downloads various models and packages and incorporated it in the docker image that is being run on serverless endpoint, i want it to run only once and download all the packages and store them for use, do i have to use network volume storage or is there another way to store the packages in container storage so they do not download every time the container is removed and started again?
11 Replies
Yes
Let me know how big are your models In total
That will be stored
If you're using network storage just download them into /runpod-volume in serverless and then check if it has been downloaded then Dont re-download , or in pods you can download it once manually to /workspace
the total download data amounts to about 25GBs
Okay yeah just use network storage
I was hoping for another solution 😩
Ah okay here is it but it's not really a solution because it's slower, Another solution is to download it when your docker is building so execute a script that downloads them when your building from the dockerfile
I would suggest creating a version with the models built into the image and another version that uses network volume and compare the response time. There is an open issue with network volumes that cause them to add delaytime to the process so you might be better of with your 25gb image.
well, the reason i went with script was to decrease the time it takes to push and deploy the image so i don't think that will work
As I said before it would be slower if your model sums up to 25gb but yeah sure try it out
With either choice you can speed up your requests by having at least 1 active worker.
and since you are only charged when your requests are being processed I suggest you set max workers to 30.
Yup but if you're using active workers ig the request will be faster if you have the models inside the image
got it, thanks!