R
RunPod•2mo ago
nayandhabarde

Need help putting 23 GB .pt file in serverless enviornment

I have a 23 GB .pt file containing tensors for 36 attention processors for each step, cannot reduce size I need to somehow put this one on the serverless environment to use in the inference, I am getting no space left on device error when bulding the docker image (I understand it should be small, but i don't have an option) Can someone please help
2 Replies
digigoblin
digigoblin•2mo ago
No space left on device when building docker image means that your local disk on the machine where you are building the image is out of disk space, it has nothing to do with RunPod serverless.
Neuraldivergent
Neuraldivergent•2mo ago
Use network storage to store the .pt file
Want results from more Discord servers?
Add your server
More Posts
Increase Spot Warning TimeI see in the docs that there is a 5s window before a spot instance is interrupted. 5s isn't really eControlNet does not seem to work on Serverless APIHey guys, before everything, I'm not a native English speaker so sorry in advance if I got somethinhow to route docker secrets to pod automaticallyI have some credentials saved as runpod secrets. After creating a new pod using runpodctl, I manuallimage deprecated?When i put all the variables in (trying with phi-4k-mini) it says image is deprecatedLora modules with basic vLLM serverlessIs it possible to use lora modules with default vLLM endpoint ? If not, how is it possible to do it runpod js-sdk endpoint.run(inputPayload, timeout); timeout not workwhen I set itmeout 360000, it always return error "timeout of 3000ms exceeded". Even I open index.ts