DannyB
RRunPod
•Created by DannyB on 11/18/2024 in #⚡｜serverless
Unable to fetch docker images
During worker initialization I am seeing errors such as:
error pulling image: Error response from daemon: Get "https://registry-1.docker.io/v2/": context deadline exceeded
2024-11-18T18:10:47Z error pulling image: Error response from daemon: Get "https://registry-1.docker.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
image pull: {repo}/{tag}: pending
failed to pull image: open /var/lib/docker/165536.165536/tmp/GetImageBlob3971008098: no such file or directory2 replies
RRunPod
•Created by DannyB on 11/11/2024 in #⚡｜serverless
🚨 All 30 H100 workers are throttled
What's going on ?
4 replies
RRunPod
•Created by DannyB on 4/26/2024 in #⚡｜serverless
A way to know if worker is persistent ("active") or not
It would be a great help if there was some way from inside the code (e.g. environment variable) to know if the model is running on a persistent worker or not.
Example use-case - If the worker is persistent, I can compile the model, it takes ~20 minutes but it is worth it so my users can get an almost 50% latency reduction. However, you can understand it is impossible to do for an ephemeral worker as it will take too long to initialize.
Is there any way to do this?
3 replies