Railway•12mo ago

Cannot Get Private Networking working for Go app and ChromaDB

ID: c6a718e0-23e7-4d39-85b8-cd951a53c53f So I have a Chroma Vector DB with that ID and I have it's PORT variable set to 8000 which is what it naively listens on. I have it as chroma.railway.internal but my app cannot seem to hit that. The strange thing is that when I enable the public networking, it can access it on the generated domain:443. I'll paste the error I'm getting down here:

error:
"Post "http://chroma.railway.internal:8000/api/v1/collections": dial tcp: lookup chroma.railway.internal on 8.8.8.8:53: no such host"
level:
"error"
time:
1702269309
@replica: 41f01962-2e35-4e48-a28b-1f107d378c6f

error:
"Post "http://chroma.railway.internal:8000/api/v1/collections": dial tcp: lookup chroma.railway.internal on 8.8.8.8:53: no such host"
level:
"error"
time:
1702269309
@replica: 41f01962-2e35-4e48-a28b-1f107d378c6f

Does anyone have some suggestions?

Solution:

try setting this as the start command

/bin/sh -c "uvicorn chromadb.app:app --workers 1 --host :: --port 8000 --proxy-headers --timeout-keep-alive 30"

/bin/sh -c "uvicorn chromadb.app:app --workers 1 --host :: --port 8000 --proxy-headers --timeout-keep-alive 30"

...

Jump to solution

35 Replies

Percy•12mo ago

Project ID: c6a718e0-23e7-4d39-85b8-cd951a53c53f,41f01962-2e35-4e48-a28b-1f107d378c6f

Brody•12mo ago

are you building with a dockerfile

OhLylnOP•12mo ago

Hey Brody! Yes i am here's my dockerfile :

FROM golang:1.21.5-bookworm

WORKDIR /app

COPY . .

ARG CHROMA_URL
ENV CHROMA_URL=$CHROMA_URL

ARG OPENAI_API_KEY
ENV OPENAI_API_KEY=$OPENAI_API_KEY

RUN CGO_ENABLED=0 GOOS=linux go build -o /biblesearch
CMD ["/biblesearch"]

FROM golang:1.21.5-bookworm

WORKDIR /app

COPY . .

ARG CHROMA_URL
ENV CHROMA_URL=$CHROMA_URL

ARG OPENAI_API_KEY
ENV OPENAI_API_KEY=$OPENAI_API_KEY

RUN CGO_ENABLED=0 GOOS=linux go build -o /biblesearch
CMD ["/biblesearch"]

I switched from the alpine build to bookworm because I saw alpine can cause issues with private networking

Brody•12mo ago

then I guess you already knew what I was going to say

OhLylnOP•12mo ago

tbh I'm not a docker expert so Idk if bookworm gets rid of alpine. I can try with a nixpack

Brody•12mo ago

try changing your CMD line to CMD sleep 3 && /biblesearch bookworm is a Ubuntu base

OhLylnOP•12mo ago

Okay cool, running that rn

Brody•12mo ago

you also don't need any of those ARG or ENV lines, that's only needed if you need a variable during build

OhLylnOP•12mo ago

Oh okay I did not know that

Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "sleep 3 && /biblesearch": stat sleep 3 && /biblesearch: no such file or directory: unknown

Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "sleep 3 && /biblesearch": stat sleep 3 && /biblesearch: no such file or directory: unknown

I am not sure if I typed that incorrectly maybe CMD ["sleep 3 && /biblesearch"]

Brody•12mo ago

show me your new dockerfile without the brackets like I wrote it

OhLylnOP•12mo ago

Okay I will try that thanks okay new error I think this is app level though maybe:

error:
"Post "http://chroma.railway.internal:8000/api/v1/collections": dial tcp [fd12:eab5:1cb0::73:9796:713]:8000: connect: connection refused"
level:
"error"
time:
1702270542
@replica: 53697e50-faf1-479f-8f5b-7660c0c49c78

error:
"Post "http://chroma.railway.internal:8000/api/v1/collections": dial tcp [fd12:eab5:1cb0::73:9796:713]:8000: connect: connection refused"
level:
"error"
time:
1702270542
@replica: 53697e50-faf1-479f-8f5b-7660c0c49c78

Brody•12mo ago

chroma needs to be listening on ::

OhLylnOP•12mo ago

Can i set that in the port env var for my chroma instance?

Brody•12mo ago

depends, does it have an applicable environment variable to set the host it listens on?

OhLylnOP•12mo ago

I don't think so I can take a loot at the docs, I am just using the default template right now I'll try it anyway

Brody•12mo ago

default template leaves a lot to be desired I'm told

OhLylnOP•12mo ago

Yeah it basically just sets up the default dockerfile and a volume it seems

Brody•12mo ago

does it deploy from a github repo?

OhLylnOP•12mo ago

No just from their github docker repo

Brody•12mo ago

well thats something at least yeah im looking at the data for the template, doesnt come with any service variables for things like public or private database url

OhLylnOP•12mo ago

Yeah it had 0 variables set which is a little frustrating because I am not sure what I can configure in it

Brody•12mo ago

yeah unfortunately railway does not yet have any quality control for templates

OhLylnOP•12mo ago

Still building I am not sure if it's just taking a long time or what just an fyi no logs either I might restart that deploy it failed anyway

Brody•12mo ago

we talking about the chroma service?

OhLylnOP•12mo ago

Yeah correct I just hit redeploy on the latest deployment Okay it's me making it fail I was setting PORT to :: on the chroma service so I think you cannot do that lol

Brody•12mo ago

:: is not a port lol its a host and the host is set to 0.0.0.0 https://github.com/chroma-core/chroma/blob/main/bin/docker_entrypoint.sh#L7

OhLylnOP•12mo ago

Hm okay I am wondering If i can set it's host to :: somehow

Solution

Brody•12mo ago

try setting this as the start command

/bin/sh -c "uvicorn chromadb.app:app --workers 1 --host :: --port 8000 --proxy-headers --timeout-keep-alive 30"

/bin/sh -c "uvicorn chromadb.app:app --workers 1 --host :: --port 8000 --proxy-headers --timeout-keep-alive 30"

Brody•12mo ago

OhLylnOP•12mo ago

deploying, I did not know you could edit start command for stuff in docker lol that is super useful promising

Brody•12mo ago

yeah opens up a lot of doors

OhLylnOP•12mo ago

I am an idiot i set that as start for my app one sec gotta switch it around THAT WORKS

Brody•12mo ago

uvicorn doesn’t support dual stack binding, so chroma will now only work on the private ipv6 network and not the public ipv4 network

OhLylnOP•12mo ago

Okay that is totally valid, thank you so much man I totally appreciate you, you've helped me before as well just wanted to give you such a shoutout

Brody•12mo ago

haha no problem, im happy to help

Gaming

Programming

Cannot Get Private Networking working for Go app and ChromaDB