Need help setting up private network w/ uvicorn and docker
I have a frontend that tries to fetch data from my backend, which is running in a docker container using uvicorn. I'm able to get everything functional if using an ipv4 address, but not if im using the private network.
My Dockerfile:
In my frontend code, i try to fetch from
backend.railway.internal/
147 Replies
Project ID:
e4f222ca-fce1-437e-a848-11bcb7c17dbd
e4f222ca-fce1-437e-a848-11bcb7c17dbd
if the users browser will be making this request you need to use the public domain
its not
im calling it internally through like a proxy in the nextjs api router
sever side rendering?
not exactly
in nextjs i have /api/py/[...path]/route.ts
it will call the python_backend_url directly
gotcha
basically proxying through directly
uvicorn doesn't support dual stack binding, so you'd either have to configure it to listen to ipv6 (internal network) or ipv4 (public network) not both, if you're okay with just listening on the internal network I can tell you how, or if you need to listen on both the internal and public I can also tell you how, your choice
yea im fine with configuring it to only internal network
ive only added the public ipv4 one bc i couldnt get it to work with the internal one
i previously had
but i dont think it worked either
sounds good, first, in the service that runs uvicorn set a PORT service variable to 3000
alright, should i change this uvicorn command to point to port 3000 as well?
nope, have you set the variable yet though?
just set, waiting for redeploy rn
how should i change my docker cmd?
there's more steps, don't bother testing just yet
alright
also ty for taking the time to help, ive already spent hours going through the very limited docs on private networks in the railway site and trying to debug this <:AA_Vanilla_Pray:905177543953498112>
your new CMD command should be this
note that I've intentionally left out the .env thing, please don't use .env files, always use service variables when on railway.
(there are still more steps)
is this the service for the python app? there's variables for next?
alr
the only variable i need to update on the frontend is
PYTHON_BACKEND_URL
and set that to the private network i believeyou have separate services for next and python right?
yes
and the screenshot above is the python service?
yea
okay cool
backend config ^
in the python service, set the
PYTHON_BACKEND_URL
to http://${{RAILWAY_PRIVATE_DOMAIN}}:${{PORT}}
cancel the redoploy popupthat variable is for the frontend
for the proxy
I know, just trust
done
once you set that, click the little eye icon on that variable to make sure it renders properly
perfect
that should be all the config needed for your python service, can you confirm you can no longer access it publicly?
uhh it hasn't been redeployed?
have you pushed the code when you changed the CMD command?
yes
then it should have redoployed/rebuilt
i pushed it before we did this change
ill repush something so it redeploys
if it has already rebuilt from the CMD command change then you're fine
just open the public domain and see what you get
it was failing the health checks
oh yeah I completely forgot about that, my bad
the health check is ipv4, so since you're listening on ipv6 only, you do actually need dual stack binding
sorry i just repushed and cancelled the other
we need to not use uvicorn anymore
so remove it from your requirements.txt
and install
hypercorn
instead, making sure it's in the requirements.txt filealrighty
hypercorn should be a near drop in replacement for uvicorn
alright
this code but uvicorn => hypercorn?
and your new CMD command with hypercorn would be
that handles dual stack binding?
indeed
alright repushed with requirements and dockerfile updated
healthcheck should now work, and the public and private domains should too
so let's move to next step
alright, ill also let u know once it finishes deploying
you have a PYTHON_BACKEND_URL service variable on the next service too right?
yes
used in this proxy code
in the next service variables, change that value to
${{backend.PYTHON_BACKEND_URL}}
and then click the eye icon to make sure that renders out properlyand that's the next service?
yes
okay then I think it should work, unless in forgetting something again (likely)
alright ill pray for it then
did your frontend just re-deploy?
will let u know once everything deploys
ah okay, sounds good
well it has to rebuild after the variable change
backend finished deploying and is working. for some reason frontend is still building...been 12 mins usually finishs in like 5...
question, for the backend, can i remove public networking entirely?
yes you can
frontend deploy literally stuck here idk
ill just wait it out and hope it'll actually finish sometime
will let u know once its done
haha just abort it
then use this to redoploy https://bookmarklets.up.railway.app/service-redeploy/
its actually building now
but that looks useful thx
so both finished deploying
but i'm getting "invalid URL" on the actual requests
again, the frontend has this
oh wait its missing a trailing slash
hold on
the url variable itself should not have a trailing slash
the way i use it in the code i use it
here
pathSuffix
should have a prefixed slash
the service variable definitely should not have a trailing slashok ill update my code to accomodate for it then
its just how its always done
ya ur right
its really weird but all the requests r now timing out? both front and backend were deployed successfully
and it passes health check as well
what is the image your next's dockerfile uses?
nextjs one uses nixpacks
what does it timeout on? can you send the error?
any api request that proxies through and tries to hit that backend
for example, https://tryrapidly.com/api/py/hi
no like what is the specific timeout error? dns lookup, conn timeout? etc
i had it running through a cloudflare proxy
i just purged cache and turned proxy off
but its still not loading anything
im looking for errors printed by next in the deployment logs
nothing at all on next side
i never disabled the public domain
so theoretically shouldn't it be accessible at https://backend-production-bc84.up.railway.app:3000/ ?
that one also times out
nope, you can only access https services externally from port 443
the first step id take would be to log the url that next requests during a proxy request
^
alr
and didnt you remove the public backend domain?
not yet
what do the logs of the python service look like?
empty
nothing relevant rlly except that its running
would anything change if we changed all the ports to port 80?
we dont wanna do that
i think prev i had issues with using port 4000 but fixed when i used ok
add
--error-logfile -
to the hypercorn start command (CMD)like this? CMD hypercorn main:app --error-logfile --bind [::]:$PORT
or with that extra -
the little dash is needed
CMD hypercorn main:app --error-logfile - --bind [::]:$PORT
alr redeployingfrontend logs, i printed out the url we try to hit (ignore the error its irrelevant to this, just had to reset cloudflare cache)
backend logs meanwhile still look like this
seems like you are running some kind of blocking code thats preventing any requests from working
hmm
there shouldnt be anything blocking
the backend code is basically unchanged since we've started
and its working locally
maybe add more workers?
--workers 4
even for endpoints that just return a hardcoded result
have you tried hypercorn locally?
not locally
trying now
after all, i did say should be a drop in replacement 😉
hypercorn main:app --error-logfile - --bind \[::\]:3000
works locally mean what really suprises me is that it does pass the healthcheck too
do you get the same logs from hypercorn locally too?
zero logs whatsoever
im not actually sure what the query-engine thing is
i dont think it should be a problem tho?
id guess the query-engine thing is the blocker here, i have seen similar things be an issue
have you tried increasing the workers?
no but do u think that could solve the problem?
i mean it was working before
also i think query-engine is some printout by prisma db
i dont think it could hurt
i add this to the hypercorn cmd right?
yep
alr redeployed once more
ur patience is impeccable wow
haha thank you
unfortunately...no change
still timing out
wait i think the server just isnt even starting...
this was from an older deploy
er wait no thats just difference between logging of uvicorn and hypercorn
yeah thats just uvicorns logs
we're so close to getting it working too
ok lemme try to isolate the problem
im going to make a fastapi server that is super simple and has just one single endpoint
and see if that works
it would
but hypercorn isnt applicable there
wait wdym?
haha sorry my bad, for some reason i thought you said flask api
nw
if this simpler version works then its def just something in code thats resulting in some blocking behavior like u mentioned
thats likely the case, though still strange that it worked with uvicorn
unauthed
but i believe you
seems like uvicorn was monkeypatching some blocking code
ok now im testing it locally and its also timing out
guess i just have some code screwed up somewhere
very interesting tho
always easier to debug locally though!
so much unfortunately the game of whack a mole continues
youll have that in the software dev world lol
Hey just wanted to let u know I ended up resolving the error. There was an await function for disconnecting from the db after making a connection that was causing it to stall
tysm brody <:HGLoli_heart:640312018511069184>
haha yep that would for sure do it, happy to help!