memory leak with DynamicVector
What is the right way to avoid memory leaks? From docs we see that DynamicVector doesn’t call elements destructors. Decorators supposed to create destructor but as far as I can tell you can’t use it manually. So what is the proper way to clean up in the and of the each step in the cycle?
from utils.vector import DynamicVector
# decorators supposed to create __del__?
@value
@register_passable("trivial")
struct Point:
var row: Int
var col: Int
var val: Int8
fn main():
for i in range(1000000):
var vec = DynamicVector[Point](1000)
for j in range(1000):
vec.push_back(Point(0,0,0))
# this code doesn’t compile:
# for j in range(1000):
# vec[j].__del__()
20 Replies
This doc page might help you - https://docs.modular.com/mojo/programming-manual.html#behavior-of-destructors
Modular Docs - Mojo🔥 programming manual
A tour of major Mojo language features with code examples.
Docs for DynamicVector state that "When it is deallocated, it frees its memory.".Your struct only has a primitive types so Mojo should know how to free each element's memory.
But it is better to get a definite answer from Mojo engineers
I agree with you and expected that, but right now my code snippet just eats memory
Congrats @zverianskii, you just advanced to level 1!
Can you provide more details? What does it mean it "eats memory"? Have you tried
vec.__del__()?That's because the vector is never dropped, if you do a time.sleep(5) you should see the memory freed.
Not familiar with details but shouldn't vector be freed after each outer loop? Or at least re-used?
Oh yeah misread the code, looks like it could be a bug @zverianskii would you mind raising an issue on github? https://github.com/modularml/mojo/issues
I think it might be. But my problem is not with memory but with even running the example. Mojo complains about wrong index. I think it comes from
push_back method. When I remove it, it runs fine.
Oh yeah that's a bug, been reported here:
https://github.com/modularml/mojo/issues/698
GitHub
[BUG]: Error with push_back() for Dynamic Vector · Issue #698 · mod...
Bug description A simple script using the push_back() function for a DynamicVector crashes with the following backtrace: mojo: /__w/modular/modular/third-party/llvm-project/llvm/include/llvm/ADT/Ar...
Congrats @Jack Clayton, you just advanced to level 11!
We actually discussed this in this thread: https://discord.com/channels/1087530497313357884/1150531576397045981. It seems like DynamicVector's initializer (unintuitively) doesn't actually set the size to the passed value, it only reserves that many elements. If you call vec.resize(0) right after initializing, it should work
Discord
Discord - A New Way to Chat with Friends & Communities
Discord is the easiest way to communicate over voice, video, and text. Chat, hang out, and stay close with your friends and communities.
I attempted to use
vec.resize(0) immediately after initialization, but nothing changed. Just to be clear: I don't have issues with creating the vector. My main concern is finding a way to trigger the destructors for the items, as DV doesn't handle this automatically. In my example, I anticipated that decorators would create a destructor for Point, which I could then call in the loop.
This worked fine for me.
Since they are register_passable you don't get to worry about destructors, and yes, DV won't handle it for you if you do have destructors defined (and if your struct is not register_passable).
Notice this error comes up if you do try to define a destructor for a register_passable struct
I am not sure why vec's memory isn't getting deallocated here, that itself is probably a bug. However, I wanted to make clear the above stuff.
If it's not a bug, then it's unintuitive that DV doesn't have a
__del__, IMO. If you want to not leak the memory, I guess you either call vec._del_old() (just found through LSP code completion, internal method), vec.clear(), or vec.data.free() (even worse IMO, you're accessing the internal pointer that DynamicVector uses).https://github.com/modularml/mojo/issues/819 created an issue, lets clarify this thing. @Stole thanks a lot for the _del_old() you saved me!
GitHub
[BUG]: DynamicVector vector in the loop doesn't call for destructor...
Bug description This will lead to a memory usage explosion. Could be solved by adding vec._del_old() at the end of each cycle, so looks like the destructor is not called. from utils.vector import D...
I think
vec.clear() is the expected or "normal" solution here.
It feels a little bad to use the internal methodsI am not sure that is a solution. It will slow the leaking but you still don’t destroy vector and create a new one. Btw
vec.clear() doesn't solve the issueHm alright, it seemed like
vec.clear() had the same effect on memory as vec._del_old(). Guess I wasn't looking closely enough.Congrats @Stole, you just advanced to level 8!