Build can't find requirements.txt
For some reason when trying to create an endpoint, it can't find the requirements when running COPY requirements.txt /app/ using Docker, although it is in the same directory as the Dockerfile itself. It only happens when using runpod and it works when using Docker locally. This is my first time doing something with Docker, so please don't roast me xD
Efficient serverless release with image caching
My current deploy system is to push a updated docker image to docker hub (with git hash as tag) and run "new release" and pass the updates docker hub instance tag. However, it does a full download of the new image (20-30gb) instead of caching. Every small update is currently very slow to roll out. Is there a more efficient way of doing deploys? Or letting it know that a docker image has been updated-in place during dev? Thank you!
Huggingface space on Serverless. How to get the Gradio API string which is the same as Worker ID?
I deployed Huggingface Space which use Gradio. If I have worker ID then I can connect to the worker usually like https://${workerID}-proxy.runpod.net/
How can I either the available workerIDs or forward my request from serverless endpoint to Gradio API which uses something like:
```...
Has the issue of slow loading models from network volumes been resolved?
1.Previously when using serverless, loading the model from a network volume from was very slow, has that issue been resolved
2. does the pod have the same issue when using a network volume....
2. does the pod have the same issue when using a network volume....
Environment Variables Crossing Serverless Endpoints
I have a 3 endpoints that use the same serverless template then I update the docker image and env vars as needed.
My issue is the environment variables seem to sync between the 3 endpoints. ie I have value X that for endpoints should be A, B, C. But after setting them I can see the endpoints all have X=C....
Charge of 50 USD failed cause I don't have enough money. Balance is 99USD. Do I need to recharge?
I currently have a balance of $99 in my RunPod account and am using a serverless endpoint. Recently, there was an attempted charge of $50 that failed due to insufficient funds. I will automatically get some money in a couple of days. But what happens if RunPod tries again to take $50 and fail? Will it stop the endpoint even if I have $99 in my runpod balance?
Embedding Model error
Hi Community!
I'm try to use Embedding models on Serverless , but no matter which model or input I'm using it's just bring me a error.
Someone have good expirence with serverless and embedding and it's works for him?...
hipaa compliance
your serverless GPU functions are awesome, but I am wondering if it is hipaa compliant
thanks...
runpodctl project deploy issue, i make file changes they aint syncing
For some reason if i make file changes now, it will still deploy, but it wont copy over my changes!
leaked shared_memory error
I am getting this error "/usr/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown\n" is anyone have solution for this?
How exactly does serverless pricing work?
When and how does it charge me? I use regular pods for A1111 but spend a lot of time idle, would I save money with serverless if I keep pods open for long periods of time without generating images?
Worker not executing job
Hi! I'm trying to deploy a simple serverless worker for PDF processing that requires GPUs. I've followed the docs and was able to build it. But, when I send a request, it's forever stuck and workers exit.
I'm having a hard time debugging it since it takes more than half an hour to build and upload the image.
Dockerfile:
...
Reusing containers from Github integration registry
I was wondering if we could reuse the containers pushed in the Runpod Registry (e.g registry.runpod.net/rp-github-build-blabla:7e1ab3844).
Use case: I wanted to create another serverless endpoint and using AS base a running endpoint's container...
embeddings endpoints
Hi, I have tried following the sparse documentation, but so far havent been able to get a non-error response or a helpful error message out of the embeddings endpoints. has anyone had any success actually using these, and if so, could you share a setup and exact request format that is known to work?
Delay time even when there are many workers available
Hi Team,
We have a serverless flow that takes less than 15 seconds and we have over 20 workers assigned to this, we routinly get delay times of 10 to 15 seconds for jobs even though there are workers sitting idle. That almost doubles our total execution time. Is there something we can do to mitigate this?...
Runpod serverless for Comfyui with custom nodes
I want to use two custom nodes in ComfyUI in runpod serverless:
ComfyUI_CatVTON_Wrapper
It requires the following dependencies:...
How to deploy ModelsLab/Uncensored-llama3.1-nemotron?
I have tried to deploy this model
https://huggingface.co/ModelsLab/Uncensored-llama3.1-nemotron
Btw I am facing cude memory issue(I have tried 24gb, 48gb), it does not work, how to fix?...
Almost no 48GB Workers available in the EU
It looks like you're getting rid of A40's. There's no EU region that offers both the A40 and A6000, that's terrible if one stores stuff on Network Volumes.
Is there more capacity coming soon?...
GitHub integration: "exporting to oci image format" takes forever.
It's been running for over 30 minutes on this step. Same image builds in less than 5 minutes in GitHub Actions.
Why does it take so long? This is the first build. Would it be better for subsequent builds (assuming there's some caching involved?)?
To me this is unusable and I much rather just do the build and push myself and just change the endpoint image version....