DeepSeek R1 Serverless for coding
I'm interested in running an FP16 DeepSeek R1 and I am wondering if Serverless is the way to go or if a Pod would be better. I need this for 2-3 hours at a time and I would like a 'dedicated' access to this environment.
Which DeepSeek R1 model should I pick (GGUF?) and how should I configure the deployment tool in Serverless to get it to run on an H100?
Thanks in advance for any help.
6 Replies
Hi i am also trying to host a deepseek r1 on serverless.
It fails at the endpoint level
Yo bro, did you found a solution ?
I don't think that anyone did. I'm trying a full pod of 4080's (5x) ... idk what else to do
Here is the detail error
Now i am trying to host using docker container, and would manually do the needful on the server side. Let's see.
maybe the endpoint didn'thave enough vram (gpu vram)