RunPod

We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!

Join

RunPod

We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!

Join

⚡｜serverless

⛅｜pods

bert

10/14/2024

How to Keep Installed Python Modules Persistent and How to Mount Multiple Volumes?

I'm running into a couple of issues on Runpod and would appreciate some help: Whenever I pause and restart my pod, all of my installed Python modules are lost. How can I make sure the Python modules I install remain persistent even after restarting? I know that this issue with persistence could probably be solved by mounting multiple volumes, but I can't find any method to mount multiple volumes in Runpod. Could you guide me on how to do this?...

zkreutzjanz

10/11/2024

Deploy pod without scheduled downtime

Trying to put up a pod i will use for a while, how can i get one that will not lose my data? Tried with many pods...

Flynn

10/11/2024

Low GPU usage

I've currently set up a pod with 4 different GPUs and allocated each GPU to a different port with the command: CUDA_VISIBLE_DEVICES=0 python main.py --listen --port 8188. It worked. I have 4 different ComfyUI tabs operating by themselves. But when I generate, the speed of generations are incredibly slow. It has taken 2+ minutes to generate a single SDXL image on all of them....

dxqbYD

10/10/2024

Special characters in pod IDs

pod IDs should not have special characters. this is meant to be used in APIs.

Flynn

10/10/2024

Why is Runpod so slow?

I'm using the RTX 6000 which has 48gb VRAM, but my generation speed on Comfy is extremely slow. Is there a reason for this?

Julian21

10/9/2024

Pod is down alltough it should be online

what can I do to quick fix this?

Flynn

10/9/2024

Why is pod speed VERY slow with multiple ongoing pods

I have created 4 separate network volumes and attached 1 pod to each of them. I've experienced VERY slow speeds when using ComfyUI. Why is this? Is Runpod limiting my VRAM because I have 4 pods going at once?

peteryoung2484

10/9/2024

H100 NVLink

If I buy two 8xH100s, can I use nvlink between multiple GPUs?

Suri

10/8/2024

Jupyter Notebook not cooperating after 1st Reboot

hey support team! chatgpt, perplexity & @ai-helper-runpod all mentioned to reach out to support... i have a comfyui/flux (1 click) template installed. everytime i restart the pod, i can still successfully login to the jupyter notebook but i cannot make any changes. if i save/upload a file, it will say not found......

new_in_town

10/8/2024

Suggest a template for this text classification model (small model from huggingface)

I want to do some (zero-shot) text classification with this model [1] or with something similar (Size of the model: 711 MB "model.safetensors" file, 1.42 GB "model.onnx" file ) Now I see a LOT of pod templates... ...

geocine

10/8/2024

Is it possible to save template overrides on the official templates?

I want to preserve my environment vars, conatainer/volume size, start commands.

Sal ✨

10/7/2024

Runpod VLLM - How to use GGUF with VLLM

I have this repo mradermacher/Llama-3.1-8B-Stheno-v3.4-i1-GGUF and I use this command "--host 0.0.0.0 --port 8000 --max-model-len 37472 --model mradermacher/Llama-3.1-8B-Stheno-v3.4-i1-GGUF --dtype bfloat16 --gpu-memory-utilization 0.95 --quantization gguf" but it doesn't work... It say "2024-10-07T20:39:24.964316283Z ValueError: No supported config format found in mradermacher/Llama-3.1-8B-Stheno-v3.4-i1-GGUF" ...

lone_samurai

10/7/2024

Creating and managing custom containers

I want to create custom containers and I have the following requirements. Please let me know how (and if) they can be achieved via runpod. I tried looking in the docs but couldn't figure out everything. 1. I want to create containers out of image hosted on Dockerhub, that are around 50GB (or more) in size -- is there a size limit on runpod? 2. I want to start new containers (with my custom image), list them and stop them from my backend -- how can this be achieved?...

Solution:

1) No problem, I have loaded 85.8GB images from Dockerhub into RunPod. 2) You can do this with the RunPod web GUI or using GraphQL scripting....

kingcozy

10/6/2024

Starting pods without having to re-run template

Hi there! Quick question for anyone that might know. I'm relatively new to runpod but have been enjoying the ease of use and I'm not sure about best practices in regards to spinning up/down pods. I've got a network volume I've been using for some comfyui work and I start a new pod everytime I sit down to start working. I use a community template to start the pod which is taking 5 minutes or so to get started since it's downloading all the models and applications needed as though it was starting...

Kyle McDonald

10/5/2024

Is it possible to run a WebRTC server on a pod?

I don't think this is possible for two reasons: 1. The Cloudflare 100 second limit means that long-running connections like video chat are not possible. 2. Ports can only be exposed individually, but WebRTC requires a whole port range to be exposed. ...

jojje

10/4/2024

Runpod API documentation

Is the runpod API documented somewhere? I've failed to find anything about it, and have had to resort to reverse engineering the webUI backend interaction (graphql mostly), and infer what the API might be by looking at how runpodctl makes API calls. Would be great to have the complete API documented, as it would allow creating much better tooling, saving users time and also increasing the overall value of the runpod platform (win-win)....

jojje

10/4/2024

Is runpodctl abandonware?

I notice there's a lot of useful PRs for runpodctl, but zero comments nor activities on those from runpod's side. So I'm wondering if there is at least someone at runpod keeping an eye on the runpodctl github project, or if it's left to rot. Would be useful to know so the community knows whether it's time to fork it or create a replacement.

Steve Cai

10/4/2024

Any updates on when network volumes will be enabled for CPU pods?

Would be super handy 🙂

Kan3ki25

10/4/2024

Docker argument issues - python sdk (docker_args)

If i copy paste the docker arguments that work fine on the website into the docker_args of runpod.create_pod method, i get error "Something went wrong. Please try again later or contact support". The only difference is that the args are in triple quotes. Im not sure how I should be formatting them....

DeanDid

10/3/2024

Modify existing FaceFusion template?

Is it possible to clone a template so that I can modify it? The template for FaceFusion Face Swapper and Enhancer is no longer being updated, and I’d like to modify it to use the updated code for version 3.0 from HuggingFace.

Previous Next

Gaming

Programming

RunPod

We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!

RunPod

We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!