spontaneous crashing due to "out of memory" when having plenty of unused memory

In a custom modpack I've been making, if I set my render distance too high (usually >8), the game spontaneously hangs and crashes. Every time this happens, it's the exact same out-of-memory error, despite me having plenty of VRAM (Minecraft uses ~3GB, my graphics card has 6GB). I've experienced this with every shader I've tried (BSL, Sildur's, Photon, Shrimple, Tea, you name it) and on multiple versions (some with just Sodium and Iris installed) and it still crashes (though the more mods I have, the faster it crashes). The game exponentially gets more unstable the more windows I have open in the background (visible or not) and also with the more RAM I allocate. Below is the part of the log where it errors out:
[21:42:39] [Thread-54/INFO]: OpenGL debug message: id=0, source=API, type=ERROR, severity=HIGH, message='Unknown internal debug message. The NVIDIA OpenGL driver has encountered
an out of memory error. This application might
behave inconsistently and fail.
(pid=5948 javaw.exe 64bit)'
[21:42:39] [Thread-54/INFO]: OpenGL debug message: id=1285, source=API, type=ERROR, severity=HIGH, message='GL_OUT_OF_MEMORY error generated. Failed to allocate memory for buffer data.'
[21:42:39] [Thread-54/INFO]: OpenGL debug message: id=1281, source=API, type=ERROR, severity=HIGH, message='GL_INVALID_VALUE error generated. Invalid size and/or offsets.'
[21:42:39] [Thread-54/INFO]: OpenGL debug message: id=1281, source=API, type=ERROR, severity=HIGH, message='GL_INVALID_VALUE error generated. Invalid size and/or offsets.'
[21:42:39] [Thread-54/INFO]: OpenGL debug message: id=1281, source=API, type=ERROR, severity=HIGH, message='GL_INVALID_VALUE error generated. Invalid size and/or offsets.'
[21:42:39] [Thread-54/INFO]: OpenGL debug message: id=1281, source=API, type=ERROR, severity=HIGH, message='GL_INVALID_VALUE error generated. Invalid size and/or offsets.'
[21:42:44] [Server thread/WARN]: Can't keep up! Is the server overloaded? Running 5258ms or 105 ticks behind
[21:42:39] [Thread-54/INFO]: OpenGL debug message: id=0, source=API, type=ERROR, severity=HIGH, message='Unknown internal debug message. The NVIDIA OpenGL driver has encountered
an out of memory error. This application might
behave inconsistently and fail.
(pid=5948 javaw.exe 64bit)'
[21:42:39] [Thread-54/INFO]: OpenGL debug message: id=1285, source=API, type=ERROR, severity=HIGH, message='GL_OUT_OF_MEMORY error generated. Failed to allocate memory for buffer data.'
[21:42:39] [Thread-54/INFO]: OpenGL debug message: id=1281, source=API, type=ERROR, severity=HIGH, message='GL_INVALID_VALUE error generated. Invalid size and/or offsets.'
[21:42:39] [Thread-54/INFO]: OpenGL debug message: id=1281, source=API, type=ERROR, severity=HIGH, message='GL_INVALID_VALUE error generated. Invalid size and/or offsets.'
[21:42:39] [Thread-54/INFO]: OpenGL debug message: id=1281, source=API, type=ERROR, severity=HIGH, message='GL_INVALID_VALUE error generated. Invalid size and/or offsets.'
[21:42:39] [Thread-54/INFO]: OpenGL debug message: id=1281, source=API, type=ERROR, severity=HIGH, message='GL_INVALID_VALUE error generated. Invalid size and/or offsets.'
[21:42:44] [Server thread/WARN]: Can't keep up! Is the server overloaded? Running 5258ms or 105 ticks behind
37 Replies
green512
green51216mo ago
What's your pagefile size set to? It sounds stupid but Windows requires pagefile backing for everything, including vram, even if it's never used. Set your pagefile to the sum of normal RAM and GPU VRAM plus a little bit extra, and I bet it will work fine.
Mango
Mango16mo ago
How do you set your pagefile (on Windows 10)? If it helps: - GPU: NVIDIA GeForce 1660 Super (overclocked w/ OC Scanner) - CPU: Intel Core i7-3770k - RAM: 16GB DDR3 - OS: Windows 10 Home (ver. 22H2)
green512
green51216mo ago
you can run the 'systempropertiesperformance' widget (Win+R then paste that) and it's on the 'advanced' tab so with 16 GB RAM and 6 GB VRAM I'd say 24 GB pagefile sounds about right
Mango
Mango16mo ago
alr this is what it's currently set to
No description
Mango
Mango16mo ago
so what should i do here mainly with the Initial size (MB) and Maximum size (MB)
No description
green512
green51216mo ago
so you can set both values to the same thing and it will have a fixed size, or you can set a min and a max. I've always used a fixed size as I want it constant, but that's up to you. if you're low on disk space, you can try setting something like min=4GB max=24GB but I would just do 24GB fixed
Mango
Mango16mo ago
alright i have plenty of drive space so i just set a fixed size
green512
green51216mo ago
And I know it sounds stupid because this will never (hardly) ever be used, but Windows just needs to know it can swap memory if it has to.
Mango
Mango16mo ago
yeah i have had vscode OOM on me while doing modpack stuff the 5 web browser-based programs don't help documentation open in firefox, vscode and discord use electron, and libreoffice is probably based on something like servo
green512
green51216mo ago
btw you can look in task manager at the 'committed' value, which is the key here. That's how much memory Windows has set aside for all programs, though they might not actually be using it. The pagefile should always be larger than that value, or else you run into trouble.
Mango
Mango16mo ago
is cpu cache a problem? for modern standards, my cpu has little to no cache
green512
green51216mo ago
nope, that's not memory in the conventional sense in that Windows doesn't allocate it
Mango
Mango16mo ago
ahhhh that's good is there a way i can prevent DWM from using so much cpu? it eats up my performance and doesn't play nice with minecraft, especially with it being so CPU-heavy and all DWM as in Desktop Window Manager
green512
green51216mo ago
Hmm, not sure about that one. Are you playing in windowed mode? DWM should get out of the way if you use fullscreen exclusive (though 'fullscreen exclusive' is kind of a misnomer as it doesn't technically exist, there is a mode that emulates it) Although that mode might have been added in Win11, I can't quite remember. link if you want to read about it: https://devblogs.microsoft.com/directx/demystifying-full-screen-optimizations/
Mango
Mango16mo ago
i've checked it with windowed mode, but i also have two monitors, so i don't know if dwm cares about that or not it also seems to still drain more cpu depending on how many windows are open (the ones that show up in the taskbar), but not on how many are visible (by that i mean not minimized)
green512
green51216mo ago
If the game is set to fullscreen and there's no Game Bar overlays, the DWM is supposed to step aside and relinquish control.
green512
green51216mo ago
No description
Mango
Mango16mo ago
good thing i have game bar disabled and both geforce experience and discord overlays
green512
green51216mo ago
easy way to tell - if you do a shortcut like Win+R, and it displays on top of the game, it wasn't in fullscreen exclusive. If it kicks you back to the desktop, you were.
Mango
Mango16mo ago
what about this though @green512 does dwm not optimize minimized windows? i've caught it at up to 60% cpu usage, even if i'm not moving around a window or doing anything crazy and i have like 3 windows open at once max
green512
green51216mo ago
I'm not sure what would cause high CPU usage. Minimized windows will still take resources as the way DWM works is every window has its own backing store (kind of like a frame buffer) that it draws to, and then they are all composited to the desktop.
Mango
Mango16mo ago
hmmmmm
green512
green51216mo ago
(As opposed to non-composited where all applications draw directly to the one frame buffer representing the desktop. But that hasn't existed on windows since ... Win7 with aero disabled?)
Mango
Mango16mo ago
i'll check the pagefile thing out (after i restart my pc) and report on results tysm
green512
green51216mo ago
hope it fixes your problem!
kristal (she) | pegasys
could you please send a log
Mango
Mango16mo ago
bruh
No description
kristal (she) | pegasys
hmm that doesn't look like a log
Mango
Mango16mo ago
i meant the massive convo before also i'm going to either close or continue the issue based on whether it stops crashing (but i have to restart my pc to do so and i'm busy rn) it works 👍
kristal (she) | pegasys
what was the fix, the dwm thing or page file? asking because the pagefile is really a bandaid solution that ignores the problem all together @Mango not to mention the larger pagefile is the slower it will be
green512
green51216mo ago
pagefile size has zero overhead except for consuming disk space And it's not a bandaid, it's what Windows requires. Swap should equal or exceed physical memory, always.
kristal (she) | pegasys
that's a really old saying that was only true when ram sizes were like 2gb
green512 ↩️
Reply to: Swap should equal or exceed physical memory, always.
kristal (she) | pegasys
nobody goes by that nowadays also yes, it is a bandaid solution because it ignores the fact that there is clearly a memory leak somewhere also the reason why pagefile will be slower is because the system has to check the entire pagefile, even if it's filled with zeroes, meaning that whatever stuff it needs to clear will take longer the larger it is
green512
green51216mo ago
It's what Windows requires. Ignore it and you will get crashes like OP. It's how it's designed. The pagefile does not have to be checked. It's not zeroed either. It's just a big file that sits on disk and does nothing, unless the kernel has to swap something out. (And obviously the kernel has efficient data structures to know what pages are stored in the file, if any. Like a quick hash table. It most certainly does not have to read the whole file, that's insane.) Also this is most certainly not a memory leak. It's Windows not being willing to allocate memory that it can't back.
kristal (she) | pegasys
but that's what a memory leak is...
green512
green51216mo ago
No it's not, not in the slightest. A memory leak would be if the kernel consumes more and more memory, and the usage increases over time, as it has lost bookkeeping records. That happening on a kernel/OS level is virtually unheard of. Memory leaks only exist at a process level. And that's not why "out of memory" is happening in the first place. It's happening because Windows will refuse to allocate memory without backing store, not because anything is actually out of memory. Adding a pagefile -- which I cannot stress enough, is not actually being used -- fixes the problem and allows Windows to allocate the memory that was already there and free and ready to go.
kristal (she) | pegasys
hm, okay
Want results from more Discord servers?
Add your server