Memory Issue ? Python Flask - Audio Processing Project - Pydub - Worker was sent SIGKILL!
Hello hello !
I am working on a Audio processing project in Python using Flask and Pydub & ffmpeg and am experiencing an issue regarding memory.
Briefly described workflow :
This app allow the user to upload an audio file or enter a Soundcloud link
The file is downloaded from Soundcloud / Or Uploaded from the user machine
Audio analysis processing step
Returning the result to the user
Removing the audio file from the server
The local version works perfectly.
The web app on railway is working like a charm for small audio file (typically up to 10mo)..
But for larger audio file I got this error : "[ERROR] Worker (pid:45) was sent SIGKILL! Perhaps out of memory?"
This is unconvenient as the goal is to make it work for file of about 200-300mo (Dj mix recording)
I'm using traces and noticed that the crash is happening after that the file is correctly downloaded.
Crash is happening at the moment I want to execute this line : ' song = AudioSegment.from_mp3(file)' creating an instance of Pydub AudioSegment from my downloaded file.
As it's working fine for small files, I would tend to say that is a memory limitation issue ?
I tried to add this to my procfile :
web: gunicorn -w 4 -t 900 -b 0.0.0.0:$PORT main:app --preload --timeout 600 && nginx -g 'daemon off;' -c <(echo "client_max_body_size 500M; client_body_buffer_size 500M;" && cat /etc/nginx/nginx.conf)
Can you think about some fixes to this issue ? π
Thank you and merry Christmas
Project id : 9355fb30-bec5-4df8-96a2-5e99f6022f58
10 Replies
Project ID:
9355fb30-bec5-4df8-96a2-5e99f6022f58- I would absolutely say this is an out of memory situation, discord tells me you are on the trial plan, trial users only get 512mb of ram, and you very likely have more than that locally, you would need to upgrade to hobby for 8gb.
- you are doing stuff with flask and python, nginx is not applicable in this context, and nginx would not even be ran because the
&& means the first command has to exit successfully before the second command is ran.
- merry Christmas!!Ok thank you Brody !
- upgrade to hobby and let me know if you are still having issues.
- remove the nginx stuff, the container for a python app doesn't even have nginx by default.
I however noticed that I never reach 512mb of memory
while you may not see a spike up to 512mb, railway does a metrics collection every minute (if I recall correctly) so it's very possible the app could try to allocate more than 512mb and then crash before metrics are collected
I do still believe this is an out of memory situation
I see ! Thank you π
Working fine on Hobby !
Awesome and welcome to the hobby plan!
Hello,
After using the app for 3 months now with Hobby Plan, I still notice my initial issue sometimes.
My app is basically processing audio files.
It is working fine for 1hour audio, but for 2hours or more I'm still getting :
[2024-03-07 20:57:35 +0000] [10] [ERROR] Worker (pid:44) was sent SIGKILL! Perhaps out of memory?
ex here : 65157f0e-fb71-46dc-b207-37e935a11c32
Is there a way I could increase memory ?
Thank you !
you can upgrade to the pro plan, once you upgrade you can transfer your project over to the pro workspace, redeploy and then your app will have access to the 32gb of ram!