3WaD
3WaD
PostsComments
RRunPod
Created by 3WaD on 2/15/2025 in #⚡｜serverless
[Solved] EU-CZ Datacenter not visible in UI
Thank you very much. It's working now. This was a small problem, probably just with my account, but you still took the time to fix it, and I appreciate it!
10 replies
RRunPod
Created by 3WaD on 2/15/2025 in #⚡｜serverless
[Solved] EU-CZ Datacenter not visible in UI
Should I make a ticket? I thought it won't be that hard to get an answer to this.
10 replies
RRunPod
Created by skeledrew on 2/18/2025 in #⚡｜serverless
Workers stuck at initializing
So there's your error. It means it couldn't pull the image because either such repo/image/tag does not exist, or you're using private repo and didn't set registry credentials in the RunPod settings.
5 replies
RRunPod
Created by skeledrew on 2/18/2025 in #⚡｜serverless
Workers stuck at initializing
What's the logs when you click on one of them?
5 replies
RRunPod
Created by Guy on 2/17/2025 in #⚡｜serverless
Serverless Docker tutorial or sample
Developing a container image for RunPod isn't that much different. You can follow Docker tutorials: https://docs.docker.com/get-started/docker-concepts/building-images/writing-a-dockerfile/ Then read through https://docs.runpod.io/serverless/overview and see how endpoints, workers and handlers work. In short: Make sure to use RunPod SDK and have a handler function as an entry point into your Python code. You can also get inspired by already existing templates. Here are the official ones from the RunPod: https://github.com/runpod-workers
2 replies
RRunPod
Created by Justin on 2/17/2025 in #⚡｜serverless
Baking model into Dockerimage
If you want to bake a model into the container image you have to do it while building it. You can't do it afterwards without rebuilding it again, which can't be done on RunPod. To download or copy a model and bake it into the image you can do: 
# Use COPY for local files on your disk 
COPY localfolder/model /containerfolder/model
# Use COPY for local files on your disk 
COPY localfolder/model /containerfolder/model
or 
#!/bin/bash
set -Eeuox pipefail

mkdir -p "$1"
cd "$1"
git init
git remote add origin "$2"
git fetch origin "$3" --depth=1
git reset --hard "$3"
#!/bin/bash
set -Eeuox pipefail

mkdir -p "$1"
cd "$1"
git init
git remote add origin "$2"
git fetch origin "$3" --depth=1
git reset --hard "$3"
# Make sure to have git installed!
COPY clone.sh /clone.sh

# Clone selected HuggingFace repo
RUN . /clone.sh /containerfolder/ https://huggingface.co/user/model
# Make sure to have git installed!
COPY clone.sh /clone.sh

# Clone selected HuggingFace repo
RUN . /clone.sh /containerfolder/ https://huggingface.co/user/model
5 replies
RRunPod
Created by Justin on 2/17/2025 in #⚡｜serverless
Baking model into Dockerimage
Why via vllm? You can do this step in the Dockerfile
5 replies
RRunPod
Created by Justin on 2/16/2025 in #⚡｜serverless
Flashboot not working after a while
Sometimes my endpoint chooses a cold worker for a request even though there's a previously used warm one showing as idle. They can't be guaranteed the same as dedicated pods. That's one of the few disadvantages we have to accept in serverless services.
19 replies
RRunPod
Created by Justin on 2/16/2025 in #⚡｜serverless
Flashboot not working after a while
Flashboot basically stays as long as the worker does. When it's shifted to a new one, you get a cold start. There's a new feature in development called Active Flashboot that should automatically prewarm the workers as they initialize. Before it's released, you can keep your workers warm only by using them or sending periodic warm requests.
19 replies
RRunPod
Created by 3WaD on 2/15/2025 in #⚡｜serverless
[Solved] EU-CZ Datacenter not visible in UI
No description
10 replies
RRunPod
Created by 3WaD on 2/15/2025 in #⚡｜serverless
[Solved] EU-CZ Datacenter not visible in UI
Nope. Please read it again. 🤭
10 replies
RRunPod
Created by blue whale on 2/11/2025 in #⚡｜serverless
Job stuck in queue and workers are sitting idle
It happens to me from time to time too. There are no logs to check because nothing is running on workers. I think it might be something with the orchestrator as the only solution is to cancel the request and send a new one. If you don't, the request is usually executed after a long time. This can randomly happen with a previously perfectly working worker.
36 replies
RRunPod
Created by AmadeusXR on 2/15/2025 in #⚡｜serverless
hey we have serverless endpoints but we have no workers for more than 12 hours now !
https://discord.com/channels/912829806415085598/1185333521124962345/1340304358784368690
8 replies
RRunPod
Created by timprepscius on 2/16/2025 in #⚡｜serverless
Costs
I don't know what is your container doing, so unfortunately I can't help you much without knowing the goal, logs or even handler code.
11 replies
RRunPod
Created by timprepscius on 2/16/2025 in #⚡｜serverless
Costs
Do you mean pulling and extracting the container image itself? That's worker initialization and it's not billed. You're billed only for the execution time of your code once you send a request.
11 replies
RRunPod
Created by timprepscius on 2/16/2025 in #⚡｜serverless
Costs
Bake everything you can into it—models, files, installations, processes, etc. You don't want to download or process anything at runtime you don't need to.
11 replies
RRunPod
Created by timprepscius on 2/16/2025 in #⚡｜serverless
Costs
Yes, the cold start is billed too. That's why you want to optimize your images as much as possible and use Flashboot.
11 replies
RRunPod
Created by AmadeusXR on 2/15/2025 in #⚡｜serverless
hey we have serverless endpoints but we have no workers for more than 12 hours now !
What data centres you have selected and how many max workers?
8 replies
RRunPod
Created by testymctestface on 12/27/2024 in #⚡｜serverless
Running worker automatically once docker image has been pulled
Yes, but only after they receive the job. And that results in the cold-start. And this happens every time the workers are shifted. If it would happen automatically on their initialization, theoretically you would never experience the cold-start on a job itself.
60 replies
RRunPod
Created by testymctestface on 12/27/2024 in #⚡｜serverless
Running worker automatically once docker image has been pulled
As I imagine, the active workers are "dedicated," which means they basically run nonstop as a pod would. That's why you pay for them this whole time. This active flash boot, on the other hand, is the same as a normal flash boot. The worker can disappear at any time since it's not dedicated to your endpoint. The only major change is that it automatically loads the models and engines on worker initialization, and that's what we want. Correct me if I am wrong.
60 replies