M
Modular•15mo ago
rd4com

Swapping pointers would cause memory problems ?

Hello, does this code would cause problems please? the motivation is to not allocate another pointer of the same size and not do a copy 
var fd = open("stories15M.bin","r")
var data = fd.read()
fd.close()

let cp_buf = data._as_ptr().bitcast[DType.uint8]()
data._buffer.data = data._buffer.data.get_null().alloc(1)
data._buffer.size = 1
data._buffer.capacity = 1
_=data
var fd = open("stories15M.bin","r")
var data = fd.read()
fd.close()

let cp_buf = data._as_ptr().bitcast[DType.uint8]()
data._buffer.data = data._buffer.data.get_null().alloc(1)
data._buffer.size = 1
data._buffer.capacity = 1
_=data
https://github.com/tairov/llama2.mojo/pull/50
1 Reply
Aydyn
Aydyn•15mo ago
Thanks @rd4com for raising this.. So , the reasoning here is to try to reuse the buffer that is already allocated for file content, to avoid double allocation. Since for example if we read full file content into memory as a next step we need to allocate some interal structures & weights. This could lead to multiple OOMs in case of users will be trying to load huge models. I have also filed a few "Feature Requests" that might help us to avoid full load OR make loading by chunks https://github.com/modularml/mojo/issues/1135 https://github.com/modularml/mojo/issues/1134 https://github.com/modularml/mojo/issues/1130

Did you find this page helpful?