How can I see the peak memory usage of a datastructure like alist in mojo?
I want to compare the memory usage of python and mojo.
For python I use the memory profiler pip package.
How can I get the accurate memory consumption of a list or other structure in mojo?
17 Replies
For a
List[T], assuming there are no pointers in T: sizeof[List[T]]() + sizeof[T]() * len(list).
The runtime hasn't opened up the kind of memory profiling that is needed to do allocation tracking on that scale.
The other option is to allocate a very big list and check the process memory usage.For Python memory profiling I would recommend Scalene https://github.com/plasma-umass/scalene as it provides data on Python and native memory usage.
GitHub
GitHub - plasma-umass/scalene: Scalene: a high-performance, high-pr...
Scalene: a high-performance, high-precision CPU, GPU, and memory profiler for Python with AI-powered optimization proposals - plasma-umass/scalene
Scalene won’t work properly with Mojo if you’re on Linux due to some of the transparent hugepage and virtual memory shenanigans that tcmalloc does there.
Right, I did not suggest to use Scalene for Mojo, but try it for Python memory profiling.
I will do it on a macbook m1 16gb ram
thanks for your suggestions
What would you recommend for mojo?
Alternatively I would use something like top to capture the memory consumption of the process
First, can you set a server nickname? I didn’t look at your name last time and we don’t allow profanity in usernames.
As for what I recommend, top might be good enough if you can make the process hand around for a while. Ideally you’d way to use a proper profiler but I have no idea what that is on MacOS, I’m only familiar with Linux and windows dev tooling.
Sorry, set it up when discord made me change my old username and i was angry. No problem.
activity monitor gives you process memory info
for a list, isn't it just capacity * element size?
In python, "element size" is a very interesting question to ask.
int takes a variable amount of memory.
So anything with an int is variable sized.python is a hellscape
Including lists.
now I see why the op was looking for a magic tool
wait, ints are boxed??
🙂
o m g
https://svn.python.org/projects/python/trunk/Objects/intobject.c
It has a vtable too!
I have to write a "paper" for a college class and my idea is to compare the memory usage of python lists with differenrt objects in it and compare it to mojo.
Maybe also compate it with numpy what would be more fair than python.
numpy vs mojo is going to be a few bytes.