acamp

PostsComments

RRunPod

•Created by acamp on 4/24/2024 in #⛅｜pods

Ollama on RunPod

@Papa Madiator and @justin [Not Staff] Thank you both for the assistance and resources! Would you happen to know if it's possible to setup llama3 on open-webui and make inferences to it using an API? I was not able to find specific instructions on how to set up an LLM on open-webui

34 replies

RRunPod

•Created by acamp on 4/24/2024 in #⛅｜pods

Ollama on RunPod

@justin [Not Staff] Hey Justin, I noticed that you were able to provide some valuable advise to other users regarding Ollama on runpod, so I was hoping to reach out to you regarding this thread, that I have yet to debug.

34 replies

RRunPod

•Created by acamp on 4/24/2024 in #⛅｜pods

Ollama on RunPod

Yes, but in this case I think it's trying to run the command "serve" along with "run" and "gemma" as the arguments.

34 replies

RRunPod

•Created by acamp on 4/24/2024 in #⛅｜pods

Ollama on RunPod

It looks like the Container Start Command can only take one command.

34 replies

RRunPod

•Created by acamp on 4/24/2024 in #⛅｜pods

Ollama on RunPod

It returns the following error: Error: accepts 0 arg(s), received 2

34 replies

RRunPod

•Created by acamp on 4/24/2024 in #⛅｜pods

Ollama on RunPod

Thank you for all the support so far, but are there any other fixes I coudl implement to get this to work?

34 replies

RRunPod

•Created by acamp on 4/24/2024 in #⛅｜pods

Ollama on RunPod

It looks like I just have to run two commands - "serve" and "run gemma", after which I should be able to make inferences with gemma, but I'm not sure how to implement that.

34 replies

RRunPod

•Created by acamp on 4/24/2024 in #⛅｜pods

Ollama on RunPod

Tried this, and the error seems to be the same.

34 replies

RRunPod

•Created by acamp on 4/24/2024 in #⛅｜pods

Ollama on RunPod

It seems to be returning the same error.

34 replies

RRunPod

•Created by acamp on 4/24/2024 in #⛅｜pods

Ollama on RunPod

If I just have "gemma", the error messages is: Error: unknown command "gemma" for "ollama"

34 replies

RRunPod

•Created by acamp on 4/24/2024 in #⛅｜pods

Ollama on RunPod

34 replies

RRunPod

•Created by acamp on 4/24/2024 in #⛅｜pods

Ollama on RunPod

Thanks for the link. I went ahead and spun up a pod with the ollama/ollama container image. After the pod starts, would you know how to make inferences with a model (e.g. gemma).

34 replies

RRunPod

•Created by acamp on 4/24/2024 in #⛅｜pods

Ollama on RunPod

I was just looking into that. Did you have any resources that maybe helpful? I was referring to this link: https://hub.docker.com/r/ollama/ollama#!, but I was wondering if there was an approach more suited to RunPod.

34 replies