Ashique A B
RRunPod
•Created by Ashique A B on 1/31/2025 in #⚡｜serverless
OSError in vLLM worker; issues when its new update was released
3 replies
RRunPod
•Created by Ashique A B on 1/30/2025 in #⚡｜serverless
Process group has not been destroyed before destruct ProcessGroupNCCL, Leaked shared_memory object
1 replies
RRunPod
•Created by Ashique A B on 12/6/2024 in #⚡｜serverless
Can anyone help me deploy a qwen/qwq-32B-Preview model from huggingface with vllm serverless
I'm having issues with configurations.
I used 1 gpu of 80gb with
container image : runpod/worker-v1-vllm:stable-cuda12.1.0.
and had set the dtype as bfloat16,
but the model is giving rubbish outputs.
1 replies