yhlong00000
RRunPod
•Created by bossman on 11/18/2024 in #⚡｜serverless
job timed out after 1 retries
Try to update your SDK to 1.7.4
11 replies
RRunPod
•Created by abtx on 11/19/2024 in #⚡｜serverless
HF Cache
Basically if you are using hugging face models and it is popular, there is a chance those models will be cached on our servers and give you a fast start
4 replies
RRunPod
•Created by abtx on 11/19/2024 in #⚡｜serverless
HF Cache
Will have more info and doc coming soon.
4 replies
RRunPod
•Created by Sergio Santos on 11/19/2024 in #⚡｜serverless
GPU Availability Issue on RunPod – Need Assistance
Hey, we have a temporary network outage at EU-RO-1, it is fixed and good now
4 replies
RRunPod
•Created by bo on 11/17/2024 in #⚡｜serverless
What are ttft times we should be able to reach?
Maybe try different GPU types? 48 pro, 80, 80 pro
2 replies
RRunPod
•Created by yasek on 11/16/2024 in #⚡｜serverless
What methods can I use to reduce cold start times and decrease latency for serverless functions
I don't have exact number, but definitely days or even weeks.
26 replies
RRunPod
•Created by yasek on 11/16/2024 in #⚡｜serverless
What methods can I use to reduce cold start times and decrease latency for serverless functions
When you first create the endpoint, the docker image will be downloaded to multiple servers, if you send request immediately, the worker isn’t ready. However after initial download finished, the image will keep on the server, unless you don’t use your endpoint for a very long time. The cold start is loading model into gpu vram, if you load model from local disk(bake model into docker image) it will faster than network volume.
26 replies
RRunPod
•Created by yasek on 11/16/2024 in #⚡｜serverless
What methods can I use to reduce cold start times and decrease latency for serverless functions
Cold start is faster when you have files baked in docker image compare to network volume. Loading file from local GPU server disk will be faster than loading from file server.
26 replies
RRunPod
•Created by zeeb0t on 11/12/2024 in #⚡｜serverless
Mounting network storage at runtime - serverless
The easiest way to deploy globally is to build all your files as part of docker image, so you don’t need network volume
11 replies
RRunPod
•Created by fireice on 11/14/2024 in #⚡｜serverless
How to Get the Progress of the Processing job in serverless ?
You can just put below code in your handler.py file, no need to know total how many steps.
runpod.serverless.progress_update(job, f"Finish step 1") "
runpod.serverless.progress_update(job, f"Finish step 2") "
runpod.serverless.progress_update(job, f"Finish step 3") "
10 replies
RRunPod
•Created by fireice on 11/14/2024 in #⚡｜serverless
How to Get the Progress of the Processing job in serverless ?
10 replies
RRunPod
•Created by octopus on 11/13/2024 in #⚡｜serverless
What is the real Serverless price?
16 replies
RRunPod
•Created by octopus on 11/13/2024 in #⚡｜serverless
What is the real Serverless price?
if you set active worker 1+
16 replies
RRunPod
•Created by octopus on 11/13/2024 in #⚡｜serverless
What is the real Serverless price?
oh, I see, that is a UI bug, will ask team to fix it. sorry for the confusion.
16 replies
RRunPod
•Created by octopus on 11/13/2024 in #⚡｜serverless
What is the real Serverless price?
can you give me a screenshot of the $0.00046/s?
16 replies
RRunPod
•Created by octopus on 11/13/2024 in #⚡｜serverless
What is the real Serverless price?
16 replies
RRunPod
•Created by octopus on 11/13/2024 in #⚡｜serverless
What is the real Serverless price?
can you give a full screenshot to show the price issue? the price should change based on how many gpus you select per worker.
16 replies
RRunPod
•Created by bossman on 11/12/2024 in #⚡｜serverless
comfy
you might want to set a higher number of max workers.
2 replies