Superintendent
RRunPod
•Created by Superintendent on 2/16/2024 in #⚡｜serverless
Deepseek coder on serverless
Hello, new serverless user here, i would be using the vllm worker, so whenever it gets spun up from a coldstart, does it have to download the model everytime? Id be running it in fp16 which means it be about 14gb of data to download
17 replies
a6000 is apparently all gone but still available on page
trying to use a6000 on community cloud but its apparently all used, which i doubt because it still shows the ability to grab one on the main gpu select page, for now i grabbed 2x3090 and am hoping that it will work for what im trying to do
6 replies