Memory Leak with my Modded Setup? OutOfMemoryError by leaving idle with chunk gen enabled
Seems DH 2.0.4-a-dev (latest commit) for Iris compat has some kind of memory leak for me and also absurdly high stuttering caused by chunk generation. Seems the cause is related to having DH do chunk generation (something wrong with my choice of DH config, or an incompatibility with C2ME [previously was unable to work alongside DH]?), but it's a slow race to an OutOfMemoryError (if it's due to any mixins conflicting with one or more optimization mods I have that fix memory leaks and optimize chunk generation, then that may be part of the issue).
Disabling DH seems to resolve the issue, as does disabling distant chunk generation while keeping DH. This is related to direct buffers memory as it appears to remain stable (with all other mods enabled) with either distant chunk generation disabled or DH disable, while keeping all other mods.
10 Replies
The particular kinds of errors:
[21:55:04] [DH-Buffer Uploader Thread[0]/ERROR]: LodNodeBufferBuilder was unable to upload buffer: Unable to allocate 1048576 bytes
F3 info (distant chunk generation disabled):
With DH enabled while in motion (creative max speed): the integrated server spikes as high as 100ms; allocation rate is about 4-5GB/s on the heap; Direct Buffers: 100s of MB/sec, quickly reaching over 1GiB; off-heap allocates at >=1MB/500ms
With DH enabled while standing still: integrated server hovers around 28ms; allocation rate is 400 - 1000MB/s on the heap; Direct Buffers: stable, fluctuates by hundreds of kilobytes, trending upwards; off-heap allocates at <=1MB/500ms
Observation: the direct buffers don't always seem to get cleared all that often until it reaches a certain threshold near 1.3GiB, and seems to not be directed by the heap and its choice of GC, but idk.
No DH while in motion (creative max speed): integrated server in [32ms, 36ms] on average with spikes to ~40-43ms; allocation rate is ~1.5-2GB/s on the heap, some stutters; Direct Buffers; stable, fluctuates by hundreds of kibibytes trending upwards; off-heap: stable, fluctuates by hundreds of kilobytes, trending upwards
No DH while standing still: integrated server in [28ms, 30ms] on average with spikes to 30-31ms; allocation rate is 400-800MB/s on the heap; Direct Buffers: stable, fluctuates by hundreds of kibibytes trending upwards; off-heap: stable, fluctuates by hundreds of kilobytes, trending upwardsAssociated full stack traces: (attached; find
[21:55:04] for relevant entries)DH config (for a config matching the one above that fails, just enable distant chunk generation): (DistantHorizons.toml attached)
Lastly, JVM info, just in case:
Java: JDK-21.0.2 HotSpot JVM
javaw.exe (running natively on Windows x64)
JVM arguments: -Xverify:none -da -dsa -XX:ReservedCodeCacheSize=512m -XX:+UseCompressedOops -XX:+DisableExplicitGC -XX:+AlwaysPreTouch -Xnoclassgc -XX:InitiatingHeapOccupancyPercent=90 -XX:NewRatio=1 -XX:SurvivorRatio=2 -XX:+UseZGC -XX:+ZGenerational
Modrinth App JVM arguments: heap memory set to 8192MB
Ok so this issue seems to kinda mostly disappear for the most part when it comes to stutters if I set the render distance low enough, but for comparison, I get virtually no dropped frames with DH disabled, 24 render distance, 24 shadow distance, and 20 simulation distance.
All that DH should be doing is rendering approximations of chunks, not even whole chunks, and even mostly just surface information, but I still take a hit for some reason, and it still mostly seems related to direct buffer usage.
integrated server still also is quite uncomfortably high; for entities to not be a comedically stuttery mess like this is a Game & Watch, I find that having 45ms as an upperbound for the integrated server suffices.interesting. there was some talk about a memory leak when playing with LOD rendering diabled but this seems to be a different issue
Also my game recently crashed from the same
LodNodeBufferBuilder was unable to upload buffer: Unable to allocate 1048576 bytes error even after increasing heap size to 16GiB.
(Is there any particular reason DH needs to allocate 1GiB every time?)
I mean it just seems to me like some sort of memory leak considering that I slowly approach an OutOfMemoryError as described above. I don't have proof of any objects on the heap being implicitly locked away again causing such leaks.
Albeit, the basis being the alarming (but still relatively slow) direct buffer allocation rate.
And just the fact that it never seems to go down.did you also do some tests with just DH?
(Direct buffer usage I have, thankfully, seen gets deallocated every now and then)
Unfortunately no, but pretty soon here, I'll be out of time as I'm really supposed to be enjoying myself right now, not in a debugging session.
lol
;-;
In any case, I can probably live with 24 render distance for now. Beats the stuttery mess that I'm getting here.