Railway•14mo ago

Help deploying ChromaDB in production?

Hi everyone. I am new to this world. I'm an ML researcher, not a software engineer. I'm trying to figure out how to deploy ChromaDB on Railway and then use it interactively from somewhere else. But no matter what I try with TCP networking, I get the following error: ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')) Code I'm using to connect on client side:

import chromadb
client = chromadb.HttpClient(
    host='HOST_NAME_PROVIDED_BY_RAILWAY',
    port='PORT_PROVIDED_BY_RAILWAY'
)
client.heartbeat()

import chromadb
client = chromadb.HttpClient(
    host='HOST_NAME_PROVIDED_BY_RAILWAY',
    port='PORT_PROVIDED_BY_RAILWAY'
)
client.heartbeat()

I also don't really understand how this whole internal private networking thing works. I'd like to create a system that can do private networking once I'm ready to move the entire app into the cloud. But I want to prototype on my local machine, so that leaves me with TCP. This is using the docker template: https://railway.app/template/tifygm The project ID: a141d396-2ac4-44e2-820c-06fd15fa7c8b Any help at all would be super appreciated. Thank you!

17 Replies

Percy•14mo ago

Project ID: a141d396-2ac4-44e2-820c-06fd15fa7c8b

Brody•14mo ago

chromadb communicates over http, not tcp. add a service variable PORT set to 8000 remove the tcp proxy and generate a railway domain for the client, the host is the railway domain and the port is 443 keep mind that this makes the database fully publicly accessible, there is no password protection take a look at this section of this article i found https://blog.amikos.tech/secure-your-chroma-db-instance-part-1-authentication-c2f1979e7c19#159c

Michael RetchinOP•14mo ago

thank you!

Brody•14mo ago

no problem

Michael RetchinOP•14mo ago

incredibly helpful It works!

Brody•14mo ago

awsome

Michael RetchinOP•14mo ago

Okay, I spoke too soon.

import chromadb
from chromadb import Settings

client = chromadb.HttpClient(
    host='chroma-production-myspecialurl.up.railway.app',
    port='443',
    ssl=True,
    settings=Settings(allow_reset=True, anonymized_telemetry=False),
    headers={'authorization': 'PASS'}
)

from chromadb.utils import embedding_functions

embedding_fn = embedding_functions.SentenceTransformerEmbeddingFunction(
    model_name='BAAI/bge-base-zh-v1.5'
)

collection = client.get_or_create_collection(
    "demo",
    embedding_function=embedding_fn
)

import chromadb
from chromadb import Settings

client = chromadb.HttpClient(
    host='chroma-production-myspecialurl.up.railway.app',
    port='443',
    ssl=True,
    settings=Settings(allow_reset=True, anonymized_telemetry=False),
    headers={'authorization': 'PASS'}
)

from chromadb.utils import embedding_functions

embedding_fn = embedding_functions.SentenceTransformerEmbeddingFunction(
    model_name='BAAI/bge-base-zh-v1.5'
)

collection = client.get_or_create_collection(
    "demo",
    embedding_function=embedding_fn
)

This all works fine, and I'm able to do things like collection.count() But when I try to add docs, it throws a 404.

collection.add(
    documents=documents[:10],
    metadatas=metadata[:10],
    ids=ids[:10],
)

collection.add(
    documents=documents[:10],
    metadatas=metadata[:10],
    ids=ids[:10],
)

Result:

HTTPError: 404 Client Error: Not Found for url: https://chroma-production-9aba.up.railway.app:443/api/v1/pre-flight-checks

...

Exception: {"detail":"Not Found"}

HTTPError: 404 Client Error: Not Found for url: https://chroma-production-9aba.up.railway.app:443/api/v1/pre-flight-checks

...

Exception: {"detail":"Not Found"}

(not my actual password)

Brody•14mo ago

thats more so a question for chroma's docs

Michael RetchinOP•14mo ago

thanks again for superhuman response time

Michael RetchinOP•14mo ago

https://github.com/chroma-core/chroma/issues/1204 It was this. Client-server mismatch. I had to load a new docker image. The one-click deploy Chroma docker image that you have is 0.4.5. Latest version is 0.4.15. I'd suggest changing it from a hardcoded version to the dockerhub chromadb, which is just chromadb/chroma. So that other people don't fall into this same issue.

GitHub

[Bug]: Collection.add is returning status 404 using Docker · Issue ...

What happened? Description: The problem arose using Chroma service within a Docker Compose environment, specifically when I tried to utilize the collection.add method. It seems that this method is ...

Brody•14mo ago

id like to note i dont work for railway, and that the template is user submitted

Michael RetchinOP•14mo ago

well then, I very much appreciate all your uncompensated labor. I hope you get some joy out of it.

Brody•14mo ago

haha thank you

jr•14mo ago

This is good feedback, thank you!

Brody•14mo ago

give conductors admin perms on templates 🙂

jr•14mo ago

Will bring up with team

Brody•14mo ago

score

Gaming

Programming

Help deploying ChromaDB in production?