Sudden gunicorn errors. App unusable.
I get various timeout and memory errors. The strange thing is that the app was running like this before with another hoster and also the first days on Railway did not cause any problems.
To explain, there are two running apps. A website and an app that converts/crops images. A calls B, B fetches the original image from A and returns an Optimized image.
This should not take long, but now I have sporadic errors that make the page unusable.
I have already tried to increase the timeout and the workers unsuccessfully.
26 Replies
Project ID:
39aa0a37-73e0-4c64-86d2-8c772db5657f39aa0a37-73e0-4c64-86d2-8c772db5657f
and
36d5ac8d-ceaa-4b8d-9561-d38745c282b4
you're on the pro plan, correct?
yes
have you moved your project over to the team?
I have tried it in both environments. At the moment the website is in the Team and the cropper in the Hobby section
and what app is having issues?
Both show the same errors. Because everything has worked, think there is some sort of networking errors between the two. And that's results in the timeout on both ends.
are you using the private network?
no. public interface
are either services using a lot of memory?
I dont think so. The stats are locking ok and i never recognized something in the past.
this works but if you visit
https://www.blacktre.es
BLACKTREES
Deine Digitalagentur in Leipzig und Köln/Bonn
Shop, Webapplikation oder Corporate Website. 🚀 BLACKTREES plant, managt und entwickelt dein Digitalprojekt! Remote und in Leipzig oder Köln/Bonn.
it will produce a lot of errors. So maybe concurrency is a problem.
but just since yesterday
yeah this is seeming like a code / config issue, maybe you wanna move to multiple threaded workers for gunicorn instead?
just adding (--threads n) to the config?
sorry I'm not well versed in python development, you would significantly benefit from reading some articles about this topic
No problem, i try it
gunicorn --timeout 120 --workers=4 --threads=4 --log-level=debug --bind 0.0.0.0:$PORT application:app
It works. Lightning fast 🙂
I know that workers need more memory and process management costs extra. I still find the result strange. Eight workers as before should be more than enough and then also the fact that it just stopped working.
before you where using a single sync worker?
startCommand = "pipenv run gunicorn --workers=2 --bind 0.0.0.0:$PORT application:app"
That is what I had, and it has worked.
After the first errors occurred, I tried increasing timeout and workers.
(by default gunicorn binds to exactly what you have specified in the bind flag)
What do you mean? There is no problem with the bind.
I mean: The request and the scaling should only need a second, or may be two. So even if the app can't do this with 2 workers in parallel, they should still fit in the timeout window one after the other.
you're right, there's no problem with the bind, it's just redundant since that is the default bind address
ok, nice that it is running now. Even if it is a bit frustrating not to be able to understand exactly what the problem was.
And thanks for your help!
that's python / gunicorn for you