How can I use ollama Docker image?
Hello. I've been trying to serve ollama on RunPod using ollama Docker image (https://hub.docker.com/r/ollama/ollama) but haven't found a way to run it.
I tried using the
docker run ... command in the Container Start Command input but I encountered an error: unknown command "docker" for "ollama". Does anyone know the correct method to use ollama on RunPod?10 Replies
You can't use docker in docker on RunPod. You have to create a template for it.
Hello, thank you for the response. If I'm unable to use ollama as shown in the screenshot, could you please tell me if there's another way to use ollama on RunPod?
Looks fine, but you may need to add
bash -c 'sleep infinity'
As the Container Start Command to ensure that your container keeps running and doesn't keep restarting.I think I'm missing something here. I thought the template is for pulling and running a Docker image. If that's the case, shouldn't I need to add the run commands from the ollama documentation (
docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama) into the Container Start Command input?No, you can't run docker in docker that won't work
Aha now I get that the Docker container is already running in this case.. In that case could you please tell me how I can set configurations such as
port before it starts running?Not familiar with that image sorry.
Okay. Thank you for your help 😀
Maybe @Madiator2011 [EU] has some time to look into it for you
you probably would need something like this
