Modular•11mo ago

Swapping pointers would cause memory problems ?

Hello, does this code would cause problems please? the motivation is to not allocate another pointer of the same size and not do a copy

var fd = open("stories15M.bin","r")
var data = fd.read()
fd.close()

let cp_buf = data._as_ptr().bitcast[DType.uint8]()
data._buffer.data = data._buffer.data.get_null().alloc(1)
data._buffer.size = 1
data._buffer.capacity = 1
_=data

var fd = open("stories15M.bin","r")
var data = fd.read()
fd.close()

let cp_buf = data._as_ptr().bitcast[DType.uint8]()
data._buffer.data = data._buffer.data.get_null().alloc(1)
data._buffer.size = 1
data._buffer.capacity = 1
_=data

https://github.com/tairov/llama2.mojo/pull/50

1 Reply

Aydyn•11mo ago

Thanks @rd4com for raising this.. So , the reasoning here is to try to reuse the buffer that is already allocated for file content, to avoid double allocation. Since for example if we read full file content into memory as a next step we need to allocate some interal structures & weights. This could lead to multiple OOMs in case of users will be trying to load huge models. I have also filed a few "Feature Requests" that might help us to avoid full load OR make loading by chunks https://github.com/modularml/mojo/issues/1135 https://github.com/modularml/mojo/issues/1134 https://github.com/modularml/mojo/issues/1130

Gaming

Programming

Swapping pointers would cause memory problems ?