Gustavo Monti
RRunPod
•Created by Gustavo Monti on 9/2/2024 in #⚡｜serverless
Llama 3.1 + Serveless
I´m trying to use this tutorial:
https://discord.com/channels/912829806415085598/1266059553838202990/1266059553838202990
Tried to use: pooyaharatian/runpod-ollama:0.0.8 and override the default start with llama3.1
but getting this error:
{
"delayTime": 16752,
"error": "model "llama3.1" not found, try pulling it first",
"executionTime": 156,
"id": "f3687a15-700f-4acf-856a-d7df024ad304-u1",
"status": "FAILED"
}
into the logs:
2024-09-02 14:52:09.063
[info]
The model you are attempting to pull requires a newer version of Ollama.
Tried to update to pooyaharatian/runpod-ollama:0.0.9 but getting some JSON decoded errors.
12 replies
How to override ollama/ollama image to run a model at startup
Hi, I´m trying to run pods using the ollama template (ollama/ollama) and trying to override the default template to during pod creating serving the model that I want.
I tried to use ./bin/ollama serve && ollama run llama3.1:8b command into "container start command" but it doesn´t work. Any way to do this? Thanks!
6 replies
REST API with Ollama
Hello everyone, I installed ollama and trying to make some request do this API using my pod instance and port and I´m getting no results or 502.
I´m using this tutorial: https://docs.runpod.io/tutorials/pods/run-ollama
4 replies