help with memory profiling FastAPI application
I'm currently facing a challenge with high memory usage by my FastAPI application hosted on Railway. My API does music source separation using the deep learning library Demucs, and the memory consumption spikes with each API request, accumulating substantial costs.
I would really appreciate if anyone here in the community have good suggestions for how I should approach memory profiling or if there are better alternative approaches to troubleshoot this!
Solution:Jump to solution
I honestly don't see much of an issue here with only the screenshots you have provided, yes there is a spike, but it does settle back down to a lower level than what it spiked to, for something that does music source separation with deep learning likely completely in memory that is not a crazy amount of memory at all
12 Replies
Project ID:
N/AN/A
Solution
I honestly don't see much of an issue here with only the screenshots you have provided, yes there is a spike, but it does settle back down to a lower level than what it spiked to, for something that does music source separation with deep learning likely completely in memory that is not a crazy amount of memory at all
@Brody I tried do memory profiling with guppy in the RQ worker, but I don't see anything unusual here either
@Brody any other ideas on how I could make requests more cost-effective? $1.60 per request does not seem very scalable to me, but I don't know what the industry standard is either, so maybe I'm being ignorant
I mean, I didn't think deep learning was really cost effective to begin with?
I think maybe that the initial memory spike is actually caused by the RQ worker downloading Demucs at runtime
I'm just baking the model into my Docker image
sounds possible
@Brody that was the issue 😅
just a friendly reminder that pinging is against the rules
oh sorry
thx for answering my questions
no problem