Custom Memory Allocator
I need to manage millions of rather long lived (minutes to weeks) byte arrays that contain hundreds of KBs in memory. I need to be able to release the arrays if the memory pressure is too high and then reuse the RAM for arrays with other sizes. It is an in-memory database , which contains hot data from multiple TB of disk storage. Since a lot of IO is involved a lot of async functions are called and I can not make sure that the thread allocating is also the thread releasing memory. I think doing this with managed memory is a bad idea, because I can not manually free stuff or do anything against fragmentation. First I just used a pool based on NativeMemory.Alloc, but since it just calls malloc which is also not optimized for my usecase, this also leads to fragmentation over a couple of weeks of runtime. Now I built my own Memory allocator based on MemoryMappedFile and it seems to work really well. My questions are: Is this a bad idea, is there a better solution and what should I pay attention to when doing this?
2 Replies
My first question would be if there are extenuating circumstances which mean you need to use a managed GC language for this, since it seems like you want the level of control more easily obtained with languages like Rust or C
Thank you for your response. I do not have any experience in C or Rust and when I started the project (4+ years ago) I did not know how deep I would have to go to achieve my goals. Also the application is a asp.net core web API and I really do like the framework (although I am sure, there are alternatives in the suggested languages). So the context is: there are 60k+ lines of code written in C# and I think in C or Rust it would be much more. The cost of a migration would be too high and everything seems to work just fine. As I understand MemoryMappedFile is a thin wrapper for mmap, which is what modern memory allocators use internally. Using MemoryManager, Memory and Span enables hiding the complexity and "low level" of the allocator. The last few .net versions also showed many performance improvements and Microsoft as a company behind it, really makes the language more interesting for me (as I use Azure to sell the software and generally work a lot in the Microsoft world).