FastAPI + Hypercorn dropping connection after 60 seconds
Hey I'm running a FastAPI with hypercorn. Everything was fine last month, now encountering server dropping requests after 60 seconds.
Remote end closed connection without response
keep-alive is 120 seconds with hypercorn. runs as expected locally but disconnects on railway after 60 seconds.
any thoughts?31 Replies
Project ID:
a321c36a-8bf1-4eba-b44d-d026fff6a67bis this a post request?
a321c36a-8bf1-4eba-b44d-d026fff6a67b
Yep
just tested a long lived post request, got well over 60 seconds, this would be a config issue with fastapi/hypercorn.
does hypercorn provide a way to configure the max request time? (keep-alive isnt the same thing)
damn, okay thanks for testing
so strange....i could run them last time i tested
and when i run local, post reqs are fine
definitely strange but not a railway issue as shown above, if you would like to test yourself, post to https://utilities.up.railway.app/upload?slimit=0.25
i will note that railway does have a 5 minute max request time for post requests.
ah okay, thanks Brody π
i'll try a fresh deploy from scratch
without any config changes i dont know how much that would do to solve this, but the good 'ol reboot cant hurt
yep fresh install has same problem - must be config π«
maybe hypercorn changed some defaults?
hmm now even tried migrating to gunicorn and uvicorn and still same issue...
this feels like an SSL / Railway issue. Still works fine on localhost
im sorry but if it was a railway issue i would be able to replicate it here
would it have something to do with nginx? can i change the config on railway? all other settings look fine in hypercorn :/
railway doesnt use nginx
what does it use? i've tried everything possible now - gunicorn, uvicorn, hypercorn. all with the same error. 60 second connection drops
apparently default nginx SSL Timeout is 60 seconds thats why i suspected
all three work on localhost
only when i deploy, it gives me the error
fastapi keeps running even when the connection drops, so i feel it has to be this right?
they currently use envoy and they have set the timeout to 5 minutes, envoy is not end user configurable.
but once again, the issue is not with railway, please try posting to that endpoint yourself
while I have provided clear proof that the timeout is not 60 seconds, i have asked a team member to confirm the timeout, they confirmed for me that it is 5 minutes
can you share the code that you're using?
I have now deployed servers with:
- flask + gunicorn
- (uvicorn | gunicorn | hypercorn) + fastapi
- expressJS
ALL have a POST timeout at 60 seconds
i'm so lost
my code was in go
are you sure this isn't a client limitation instead? how are you testing these timeouts
hmm might be.... testing with python
requestsI tested with the insomnia desktop app
requests.post(url, headers=headers, data=json.dumps(data))how much json are you sending if it's taking more than 60 seconds lol
also no idea what i'm looking at here but this feels sus to me
https://github.com/railwayapp/nixpacks/blob/33bbd27224604ae882b66895dfa52c9cbe5c986a/src/providers/staticfile.rs#L120
GitHub
nixpacks/src/providers/staticfile.rs at 33bbd27224604ae882b66895dfa...
App source + Nix packages + Docker = Image. Contribute to railwayapp/nixpacks development by creating an account on GitHub.
i'm just calling
sleep(65) on my end to testthat has nothing to do with the python or node providers
some requests in prod take longer than 60 seconds
timeouts look fine for both locally
might try deploy with another cloud provider to test...
I don't know what to tell you here, I have proven that it isn't a limitation with Railway, and a railway team member has told me directly that it's 5 minutes
I can show you that message if you'd like?
okay thanks Brody nah its fine
thanks for trying to help π
no problem!
Solved! Something in the nix-built dockerfiles is dropping posts after 60 seconds. Deployed via my own dockerfile and weβre cooking π¨βπ³ thanks for your help again Brody!