muggleborntribute#0
RRunPod
•Created by muggleborntribute#0 on 3/17/2025 in #⚡｜serverless
Serverless docker image deployment
Hi,
I finetuned a lora from llama 3.2 3B using unsloth. and want to deploy that on serverless.
Using vLLM with merged model degrades the performance too much to be of use. I then, followed instructions from this link https://github.com/runpod-workers/worker-template/tree/main and created a serverless endpoint using the docker image. but it keeps on initializing and does not complete one job. job remains in queue.
I might be missing something. I also don't have much experience with docker. I might be making a mistake there. But I did test the docker locally before deploying. I would appreciate any help regarding this.
8 replies