Help! Why do some of my workers report insufficient space when pulling images?
Hello everyone, I have a question for you. As shown in the picture, some of my workers will report insufficient space, but most of them will not.
How to deploy Multi-Modal Model on Serverless
I am trying to deploy meta-llama/Llama-3.2-11B-Vision (which is just 11B model) on vllm serverless . Using
M = (P * 4B) / (32/Q) * 1.2 this formula ,I estimated that 26GB of VRAM should be enough to deploy it. But i tried hosting it with 48GB, 80GB, also tried allocataing two 48GB VRAM per node, but it never works. I am getting this error torch.OutOfMemoryError.
Buil docker with environment variables
Hi,
I try to build a docker from github repo:https://github.com/weaviate/multi2vec-clip-inference. And I setup env variables but I have the error:
2025-03-31 18:51:46 [INFO] > [8/9] RUN ./download.py:...
Hi!Currently, the serverless service I created keeps initialzing. Is this normal?
Endpoint ID
wi7hpbtrbmc6qh...
Unexpected Infinite Retries Causing Unintended Charges
I recently ran my serverless workload using my custom Docker image on RunPod, and I encountered an issue that resulted in significant unexpected charges. My application experienced failures, and instead of stopping or handling errors appropriately, it kept retrying indefinitely. This resulted in:
- $166.69 charged by OpenAI due to repeated API calls.
- $14.27 charged on RunPod for compute usage....
Serverless vLLM workers crash
Whenever I create a serverless vLLM (doesn't matter what model I use), the workers all end up crashing and having the status "unhealthy". I went on the vLLM supported models website and I use only models that are supported. The last time I ran a serverless vLLM, I used meta-llama/Llama-3.1-70B, and used a proper huggingface token that allows access to the model. The result of trying to run the default "Hello World" prompt on this serverless vLLM is in the attached images. A worker has the status...
