DannyB
RRunPod
•Created by DannyB on 12/9/2024 in #⚡｜serverless
Structure of "job" JSON
I understand that at the very least there is
job["id"] and job["input"] and we utilize it.
It will help me a great deal if I could send additional information like job["source"] or other metadata to the handler function.
It seems like no matter how I structure the JSON, only id and input end up in the job JSON to the handler.
Is this indeed the case?
I prefer not to force additional metadata inside the input field ... I want a separation between the model's inputs and the handler controllers.10 replies
RRunPod
•Created by DannyB on 11/18/2024 in #⚡｜serverless
Unable to fetch docker images
During worker initialization I am seeing errors such as:
error pulling image: Error response from daemon: Get "https://registry-1.docker.io/v2/": context deadline exceeded
2024-11-18T18:10:47Z error pulling image: Error response from daemon: Get "https://registry-1.docker.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
image pull: {repo}/{tag}: pending
failed to pull image: open /var/lib/docker/165536.165536/tmp/GetImageBlob3971008098: no such file or directory10 replies
RRunPod
•Created by DannyB on 11/11/2024 in #⚡｜serverless
🚨 All 30 H100 workers are throttled
What's going on ?
4 replies
RRunPod
•Created by DannyB on 4/26/2024 in #⚡｜serverless
A way to know if worker is persistent ("active") or not
It would be a great help if there was some way from inside the code (e.g. environment variable) to know if the model is running on a persistent worker or not.
Example use-case - If the worker is persistent, I can compile the model, it takes ~20 minutes but it is worth it so my users can get an almost 50% latency reduction. However, you can understand it is impossible to do for an ephemeral worker as it will take too long to initialize.
Is there any way to do this?
3 replies