RunPod

R

RunPod

We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!

Join

⚡|serverless

⛅|pods-clusters

environment variables

how to get environment variables while build? I set variables in endpoint settings, but they are not available during build. Please tell me if I missed something? ...
Solution:
You can either build the Docker image yourself, push to a docker registry and deploy from there. Or if you want to use the Github Integration you need to put your key inside the Dockerfile, this is the current solution.
No description

Network Volumes for Custom Models with ComfyUI Serverless"

Hey everyone, I'm trying to understand network volumes with RunPod serverless. If I create a ComfyUI serverless endpoint with the default worker image and attach a network volume, can I then launch a regular ComfyUI pod using that same volume to add my custom models and workflows? And will those custom models then be accessible when my serverless endpoint runs? Basically trying to extend the default image without modifying it.

build error (wrong path)

Hi, I think there is a bug in the building pipeline. I have set "Dockerpath" to "xx/yy/Dockerfile" in my serverless config. But when building i got an error: ```...

Change headers of webhook call

I would like to add an access token to the webhook call. In addition my hosting service blocks requests with GO as user-agent 🤷...

Seeing a bunch of running workers but none of them is running jobs.

Hey I experienced a lot of serverless workers running but none of them is picking request to run jobs other than the cost

Regular "throttled" status

Hi, I've configured a serverless endpoint with the max_workers setting explicitly set to 1. I've observed that the single worker for this endpoint frequently enters and stays in the "Throttled" state. This seems to be causing significant delays in request processing, making them take much longer than the actual inference time. ...
Solution:
when you set max worker to 1, your worker only deploy to single machine, when you not using it, we will give that machine to other people and when machine is fully used, your worker will be throttled. Highly suggest to avoid set max worker to be 1.
No description

Error running ComfyUI workflow from pod on serverless

I’m encountering an error when running a ComfyUI workflow from a pod on RunPod serverless. Previously, I was running the ComfyUI workflow on a pod with network storage mounted, and it worked fine. Now, I want to run the workflow via API, so I deployed the endpoint using the image: timpietruskyblibla/runpod-worker-comfy:3.4.0-base with my network storage mounted as the endpoint’s storage....
No description

Requests stuck in queue

Hi I am having issues with my serverless deployment - tasks are stuck in queue for 6-10 min, while there are idle workers (screenshot 1) I believe the issue to be with how the container is started, and not with the image itself....
No description

Disk Volume Pricing for Serverless

I'm looking for clarification on disk pricing for serverless workers. On the pricing page a Container Disk price for running pods of $0.10/GB/month (and $0.20/GB/month for idle pods). How does this translate to the serverless workers? When I create a template for my endpoint I specify Volume Disk (e.g 20Gb); how am I being charged for this? 20 * $0.20 *number of workers per month (assuming the workers are idle)? ...

Requests on Runpod Execute Twice, Doubling Billing Costs

I am experiencing unexpected behavior when running BerTopic clustering jobs. After the job finishes and posts the results, it unexpectedly triggers another execution of the same request. This results in the job running twice, effectively doubling my costs. This issue occurred previously, but it was resolved. However, it has now reappeared....

Failed to get job

im getting this ```python {"requestId": null, "message": "Failed to get job. | Error Type: ClientResponseError | Error Message: 502, message='Bad Gateway', url='https://api.runpod.ai/v2/[HIDDEN_ID]/job-take/3aa8fhul0on7fh?gpu=NVIDIA+A40&job_in_progress=0'", "level": "ERROR"} {"requestId": null, "message": "Failed to get job. | Error Type: ClientResponseError | Error Message: 502, message='Bad Gateway', url='https://api.runpod.ai/v2/[HIDDEN_ID]/job-take/3aa8fhul0on7fh?gpu=NVIDIA+A40&job_in_progress=0'", "level": "ERROR"}...

installing and using extensions on Automatic 1111

Hi everyone,
I'm trying to figure out how to install extensions on my panel via RunPod Serverless.
Since it looks like the panel needs a restart after adding an extension to apply the changes, I'm not sure how to handle that properly in a Serverless setup.
Also, for features like Refactor or using other specific options that don't seem to have direct API endpoints, is there any recommended way to interact with them?
It feels like there’s no clear method for these cases, and I'm a bit lost. ...

Stuck when run is triggered via API call but not on dashboard?

I have a project that let's me upload videos on google cloud storage (it is very bare and that's the only thing that it does at the moment). If I trigger the request form serverless dashboard, the job gets completed, but if it is triggered via API it is stuck forever this is what the code looks like:...
No description

No Space Left on Device /var/lib/docker/tmp reported during Worker Initialization

I am seeing "no space left on device" failure when initializing a serverless worker, RTX 4090 / 41 GB RAM class in US-IL. Does this mean that the worker does not even have enough disk space to deploy my Docker image?
-- snip -- 69168d8a856c Extracting [==============================================> ] 1.83GB/1.96GB 69168d8a856c Extracting [===============================================> ] 1.846GB/1.96GB...

Questions About Running ComfyUI Serverless on RunPod

I set up my ComfyUI project, ComfyUI Manager, custom nodes, and models on RunPod inside the /workspace directory of my network volume. When I temporarily deploy the volume and run python main.py --listen, I can access my ComfyUI workflow through the web on RunPod and generate images without any issues. However, after spending a few days trying to figure it out, I still can’t get it working with the serverless API. I've gone through a bunch of docs and videos, but to be honest, I'm just more confused now. The workflow runs perfectly through the web but I could never get it to run through serverless. Since everything is working fine on the web version, I feel like I'm really close to getting it working through the serverless API too. I'd really appreciate any help with this. I can also send over my files via DM if needed....

ComfyUI: "Failed to connect to server at http://127.0.0.1:8188 after 500 attempts" on serverless

Hi everyone, help would be greatly appreciated! 🙂 We're trying to move from permanent Pods to serverless and ran into this brickwall. We're having a sales call with Runpod on Monday so it's time sensitive. I followed the official instructions at https://github.com/runpod-workers/worker-comfyui . I've opened the port on the serverless endpoint but it does not solve the issue. We're using the Dockerfile from the official repo with slight modifications. Any ideas?...
Solution:
Thank you @Jason for the help! I had to tweak COMFY_API_AVAILABLE_MAX_RETRIES in rp_handler instead, but it did resolve the issue

US-NC-1 Failing to pull images

Just an FYI - Constantly having to kill these ones as they get stuck in Initializing error pulling image: Error response from daemon: Head "https://registry-1.docker.io/v2/runpod/worker-v1-vllm/manifests/v2.4.0stable-cuda12.1.0": Get "https://auth.docker.io/token?scope=repository%3Arunpod%2Fworker-v1-vllm%3Apull&service=registry.docker.io": read tcp 172.19.7.13:37010->98.85.153.80:443: read: connection reset by peer Worker ID - 45hzf7q7kf58sy...
Next