RRunPod•Created by Igor Gulamov on 2/11/2025 in #⛅｜pods deepseek-r is loading for >1h into vram.
I use my own docker container with sglang inside. For rocm you have only pytorch, no vllm or sglang.
I use loaded to the disk model. There is no space to load the 2nd one: 670gb total. and it is downloading for 4 hours, not for 1
