moonlight
RRunPod
•Created by moonlight on 1/24/2024 in #⚡｜serverless
question about the data structure of a serverless endpoint
Thank you very much!
24 replies
RRunPod
•Created by moonlight on 1/24/2024 in #⚡｜serverless
question about the data structure of a serverless endpoint
As static you mean they will not be replaced frequently, right? yes, they will be static, and yes i'm thinking on a 48gb gpu. At the moment i'm building the project and your help is very useful to setup everything on the proper way
24 replies
RRunPod
•Created by moonlight on 1/24/2024 in #⚡｜serverless
question about the data structure of a serverless endpoint
The plan is to put SDXL + Vision Language Model and eventually some other smaller model, running them on a chain. I estimate about 36gb vram, I will need to start, do the work and inmediatly stop, so i'll need to load to vram on each request. The first model to run will be the sdxl so maybe it's possible to fit it in the docker image and the rest on the network volume, what do you think about?
24 replies
RRunPod
•Created by moonlight on 1/24/2024 in #⚡｜serverless
question about the data structure of a serverless endpoint
Thank you for the information
24 replies
RRunPod
•Created by moonlight on 1/24/2024 in #⚡｜serverless
question about the data structure of a serverless endpoint
This happens just with the first request, right? After that a cold-start will not download again the image?
24 replies
RRunPod
•Created by moonlight on 1/24/2024 in #⚡｜serverless
question about the data structure of a serverless endpoint
Okay, and how much is the impact (approx) of starting a 10gb image in comparison of a 1gb image? (loading both the same packages during startup)
24 replies