codyman4488

PostsComments

RRunPod

•Created by codyman4488 on 3/4/2025 in #⚡｜serverless

how to run a quantized model on server less? I'd like to run the 4/8 bit version of this model:

https://huggingface.co/unsloth/DeepSeek-R1-Distill-Qwen-32B-GGUF

5 replies