NERDDISCO Posts - Answer Overflow

NERDDISCO

•Created by NERDDISCO on 8/9/2024 in #⚡｜serverless

Slow network volume

Some people reported, that loading models from network-volumes is very slow compared to baking the model into the image itself.

64 replies

RRunPod

•Created by NERDDISCO on 7/25/2024 in #⚡｜serverless

Llama 3.1 via Ollama

19 replies

RRunPod

•Created by NERDDISCO on 1/16/2024 in #⛅｜pods

CUDA 12.3 support

I created a template with a custom image (based on runpod/containes) to run CUDA 12.3, but when I use pytorch 2.1.2 + python 3.10, it tells me that it's not working.

python3 -c "import torch; print(torch.cuda.is_available())"

CUDA initialization: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 804: forward compatibility was attempted on non supported HW (Triggered internally at ../c10/cuda/CUDAFunctions.cpp:108.)
  return torch._C._cuda_getDeviceCount() > 0
False

python3 -c "import torch; print(torch.cuda.is_available())"

CUDA initialization: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 804: forward compatibility was attempted on non supported HW (Triggered internally at ../c10/cuda/CUDAFunctions.cpp:108.)
  return torch._C._cuda_getDeviceCount() > 0
False

The same docker image works locally on my machine, so I assume this is something on your side or am I wrong?

82 replies

Gaming

Programming