baldy

PostsComments

RRunPod

•Created by acamp on 11/12/2024 in #⛅｜pods

Ollama on Runpod

gl

14 replies

RRunPod

•Created by acamp on 11/12/2024 in #⛅｜pods

Ollama on Runpod

14 replies

RRunPod

•Created by acamp on 11/12/2024 in #⛅｜pods

Ollama on Runpod

i'd also suggest one other change. create a disk (network ideally if you can) of some size and mount it in /root/.ollama so ollama data survives stops/starts.

14 replies

RRunPod

•Created by acamp on 11/12/2024 in #⛅｜pods

Ollama on Runpod

for what it's worth ghcr.io/open-webui/open-webui:0.3.35-ollama comes with ollama 0.3.14, on which i'm running llama 3.1 70b

14 replies

RRunPod

•Created by acamp on 11/12/2024 in #⛅｜pods

Ollama on Runpod

it's real easy to get the specific version of ollama using ollama/ollama. it's more complicated with open-webui because you need to figure out what version of ollama was packaged with a version of open-webui.

14 replies

RRunPod

•Created by acamp on 11/12/2024 in #⛅｜pods

Ollama on Runpod

runpod's instructions are more complicated than is necessary these days, imo. i just use ollama or openwebui images, so either ollama/ollama:0.x.y or ghcr.io/open-webui/open-webui:0.x.x-ollama (if you go with openwebui, you should open ports 11434 and 8080, otherwise 11434 is enough).

14 replies

RRunPod

•Created by acamp on 11/12/2024 in #⛅｜pods

Ollama on Runpod

i'm also using ollama in some runpods and haven't had too many problems, although i am using streaming (for the most part)

14 replies

RRunPod

•Created by acamp on 11/12/2024 in #⛅｜pods

Ollama on Runpod

i think 524 is an error generated by cloudflare when the connection times out -- are you requesting streamed responses? if not, i wonder if the response is just taking too long and maybe the proxy is cutting off the connection due to inactivity

14 replies

RRunPod

•Created by K-Woww on 10/31/2024 in #⛅｜pods

ComfyUI: Diagnosing errors like "Syntax error: Unexpected token '>'" by logging to file?

often it's like a 502, but it's up to CF

7 replies

RRunPod

•Created by Ammar Ahmed on 10/31/2024 in #⛅｜pods

Network Volume as Storage for images

additionally, cpu pods can't use network volumes

3 replies

RRunPod

•Created by K-Woww on 10/31/2024 in #⛅｜pods

ComfyUI: Diagnosing errors like "Syntax error: Unexpected token '>'" by logging to file?

(i suspect you're getting a HTML response, btw, probably from cloudflare)

7 replies

RRunPod

•Created by K-Woww on 10/31/2024 in #⛅｜pods

ComfyUI: Diagnosing errors like "Syntax error: Unexpected token '>'" by logging to file?

open the tools (F12 or option-command-J in chrome), go back to the browser, run the workflow, and then go back to dev tools. look at the network tab in the tools to find the raw request and response. there isn't a way to get the output otherwise.

7 replies

RRunPod

•Created by K-Woww on 10/31/2024 in #⛅｜pods

ComfyUI: Diagnosing errors like "Syntax error: Unexpected token '>'" by logging to file?

you'd need to look at your browser's developer tools

7 replies