Next Image and Caching. High RAM usage.
Hi, recently i've been looking more closely at my RAM usage on my nextjs droplet. And i've noticed that the RAM on a fresh instance of the app sits at around 150mb (on the node process), but after some time and clicking around on the website it starts racking up to as much as 1.4GB/1.5GB I have available - this dramatically slows down everything. I turned image optimization off, and it never got higher than 300mb with it off.
Is this normal? Am I doing something wrong?
22 Replies
Wonders of caching
Next needs to cache the images somewhere to avoid lookups
Ye but it shouldn't fill that much RAM up should it??? Like that's actually nuts.
I have a lot of images, but still. Is there no way to use optimization but still keep my RAM usage low?
Unknown User•2y ago
Message Not Public
Sign In & Join Server To View
Not an option. I have over 20k images
Unknown User•2y ago
Message Not Public
Sign In & Join Server To View
Hey thanks for the idea. I had already read through this but read this:
The expiration (or rather Max Age) of the optimized image is defined by either the minimumCacheTTL or the upstream image Cache-Control header, whichever is larger.
Where can I check my upstream image Cache-Control header?
I had already messed around with the TTL but didn't get any results out of it. So i'm wondering if the upstream header is set to something highUnknown User•2y ago
Message Not Public
Sign In & Join Server To View
I'll try that thank you
Yeah the cache control from the img is
So i'll try putting a TTL to see if my RAM get's reduced after some time
Yeah made zero difference putting a TTL of 60s
My RAM still stays the exact same after rendering in images
Unknown User•2y ago
Message Not Public
Sign In & Join Server To View
Ah yeah, that won't fix my problem then. The problem is once an image is called it is stored permanently in my server's RAM
Which I don't want as I have over 20k images, and if every image takes up 700kb of RAM, my RAM will be filled after 3K images
Unknown User•2y ago
Message Not Public
Sign In & Join Server To View
Yeah but will this really be the solution to my problem?
Another thing I have noticed. When I go through a page with 500 images the following happens:
Optimized - true ->
RAM starts out at 150mb before opening page.
While scrolling through, my CPU will hit around 83% usage consistently,
Images will load in pretty slow, as in 2-3sec per image.
RAM usage will start skyrocketing. After loading in all 500 images RAM hits 900mb. This stays forever until server restarts, not a single MB gets dropped.
Optimized - false ->
RAM starts out at 150mb before opening page
While scrolling through, CPU is around 60% consistantly (less than optimized images)
Images will load in extremely fast, in split second. Even though resolution loaded in is 1700x1100 instead of ~400px
RAM usage will go pretty high, after all 500 images loaded, around 700mb. But after around a minute or two, it will dip down back to ~150mb.
The overall performance of optimized images seems absolutely terrible to me? even tuogh they are being loaded in at a lower resolution, my CPU usage is way higher, and my RAM skyrockets permanently, images also get loaded in WAY slower
Unknown User•2y ago
Message Not Public
Sign In & Join Server To View
Hmm. I'll try putting in less deviceSizes. I'm gonna try with deviceSizes: [640, 1080, 1920, 3840].
But yeah, I understand more CPU is needed to optimize these, and that isn't really a problem if it only happens once per image (which it does so thats fine). But the RAM usage is bonkers to me. I have 27k images. After 500 image the RAM usage is already +800mb. That means after 27k i'd need 27GB RAM
That's just not doable
Unknown User•2y ago
Message Not Public
Sign In & Join Server To View
I'll look into that. Thank you. Can confirm though that with less deviceSizes RAM usage is indeed down. Managed to almost half the RAM usage. 13GB RAM is still a lot tho for all my images haha.
Has nobody run into these issues with next/image before though? Or do most people not serve so many images? Our usecase is an ecommerce website.
Unknown User•2y ago
Message Not Public
Sign In & Join Server To View
I see
I just wish there was a way for the images to be thrown out of RAM after a while of them not being accessed.
Unknown User•2y ago
Message Not Public
Sign In & Join Server To View
Yeah i'll mess around with it. And worst case i'd have to just restart my webserver on a scheduled basis. For example every 3 days. A simple restart takes like 5 seconds only, so that shouldn't be that big a deal. And that'll clear out the RAM. Downside is though, it'll also clear out the images that I actually want to be kept in the RAM (Like frontpage etc.)
Thanks for your help
An update on this by the way if someone wants to chime in.
It seems that after optimizing the images for the first time, and then restarting the server, the images will remain optimized (as theyre cache'd in .cache/images), thus the CPU load will not happen twice, interestingly enough though, the RAM usage after a restart is extremely low, comparable to having image optimization off. But the images remain optimized
Might there be a memory leak in image optimization?
Goes from 700mb to 350mb after a restart on 500 images, and images remain optimized the same way as before.