Modular•5mo ago

What UnsafePointer can point to and allocate mem for?

I opened an issue last week in which I reported a problem with allocating for an array of Sets https://github.com/modularml/mojo/issues/2503 This issue was closed as not planned'which leaves me confused right now,. From the docu it says

UnsafePointer is a pointer type that can point to any generic value that is movable.

and Set in this example seems to be an appropriate object to point to.
As usual, I guess there is a fundamental misunderstanding here on my side and I wonder if anybody can explain to me, what Unsafepointer can point to and allocate memory for. The docu says

T (AnyType): The type the pointer points to.

but Set comfirms with AnyType of course ... Thx

GitHub

[BUG] Accessing allocated memory for an array of sets crashes · Iss...

Bug description the following code crashes: from collections import Set @value struct IDStruct(KeyElement): var id: Int fn eq(self, other: Self) -> Bool: return self.id == other.id fn ne...

10 Replies

Ehsan M. Kermani (Modular)•5mo ago

Good question! First an important point is try to avoid unsafe ptr as much as possible one reason is accessing an uninitialized ptr through ptr[0] is an Undefined Behaviour (UB). Also note that ptr[0] returns a reference and by definition, Mojo Reference is a non-nullable safe reference so ptr[0] is accessing an uninitialized reference which might be null so it's UB. To safely store values, you need to use __get_address_as_uninit_lvalue . Here's how it'd look like

from collections import Set
from testing import assert_true

@value
struct IDStruct(KeyElement, Stringable):
    var id: Int

    fn __eq__(self, other: Self) -> Bool:
        return self.id == other.id
    fn __ne__(self, other: Self) -> Bool:
        return self.id != other.id
    fn __hash__(self) -> Int:
        return hash(self.id)
    fn __str__(self) -> String:
        return str(self.id)

fn main() raises:
    var ptr = UnsafePointer[Set[IDStruct]].alloc(10)
    __get_address_as_uninit_lvalue(ptr.address) = Set[IDStruct]()
    assert_true(ptr[0] == Set[IDStruct]())
    # second element through offset like
    __get_address_as_uninit_lvalue((ptr + 1).address) = Set[IDStruct](1)
    assert_true(ptr[1] == Set[IDStruct](1))

from collections import Set
from testing import assert_true

@value
struct IDStruct(KeyElement, Stringable):
    var id: Int

    fn __eq__(self, other: Self) -> Bool:
        return self.id == other.id
    fn __ne__(self, other: Self) -> Bool:
        return self.id != other.id
    fn __hash__(self) -> Int:
        return hash(self.id)
    fn __str__(self) -> String:
        return str(self.id)

fn main() raises:
    var ptr = UnsafePointer[Set[IDStruct]].alloc(10)
    __get_address_as_uninit_lvalue(ptr.address) = Set[IDStruct]()
    assert_true(ptr[0] == Set[IDStruct]())
    # second element through offset like
    __get_address_as_uninit_lvalue((ptr + 1).address) = Set[IDStruct](1)
    assert_true(ptr[1] == Set[IDStruct](1))

Ehsan M. Kermani (Modular)•5mo ago

To know more about __get_address_as_uninit_lvalue see https://docs.modular.com/mojo/changelog#week-of-2023-04-17

Mojo🔥 changelog | Modular Docs

A history of significant Mojo changes.

Ehsan M. Kermani (Modular)•5mo ago

Also for completeness, another option currently is via initialize_pointee_move which becomes

ModularBot•5mo ago

Congrats @Ehsan M. Kermani, you just advanced to level 6!

Ehsan M. Kermani (Modular)•5mo ago

from collections import Set
from testing import assert_true
from memory.unsafe_pointer import initialize_pointee_move

@value
struct IDStruct(KeyElement, Stringable):
    var id: Int

    fn __eq__(self, other: Self) -> Bool:
        return self.id == other.id
    fn __ne__(self, other: Self) -> Bool:
        return self.id != other.id
    fn __hash__(self) -> Int:
        return hash(self.id)
    fn __str__(self) -> String:
        return str(self.id)

fn main() raises:
    var ptr = UnsafePointer[Set[IDStruct]].alloc(10)
    initialize_pointee_move(ptr, Set[IDStruct]())
    assert_true(ptr[0] == Set[IDStruct]())
    # second element
    initialize_pointee_move(ptr + 1, Set[IDStruct](1))
    assert_true(ptr[1] == Set[IDStruct](1))

from collections import Set
from testing import assert_true
from memory.unsafe_pointer import initialize_pointee_move

@value
struct IDStruct(KeyElement, Stringable):
    var id: Int

    fn __eq__(self, other: Self) -> Bool:
        return self.id == other.id
    fn __ne__(self, other: Self) -> Bool:
        return self.id != other.id
    fn __hash__(self) -> Int:
        return hash(self.id)
    fn __str__(self) -> String:
        return str(self.id)

fn main() raises:
    var ptr = UnsafePointer[Set[IDStruct]].alloc(10)
    initialize_pointee_move(ptr, Set[IDStruct]())
    assert_true(ptr[0] == Set[IDStruct]())
    # second element
    initialize_pointee_move(ptr + 1, Set[IDStruct](1))
    assert_true(ptr[1] == Set[IDStruct](1))

Martin Dudek•5mo ago

Thank you so much @Ehsan M. Kermani for all your efforts to explain, I would have been completely lost without this. I think i understand and could use this knowledge but what would be the recommended way to implement this without need of these more complicated steps. My problem with Listis that it needs CollectionElement type and i just dont need all the dynamic aspects of List, it seems not the right contruct. ÌnlinedFixedVector would be much more what i am looking for but its limited to AnyRegType What I actually need this for is for parallelization, to go along with the example, I need a Set for each individual work_item process and after parallelization i combine the sets in whatever way. Thanks

Arthur Evans•5mo ago

Any @value struct should qualify as a CollectionElement I believe. Oh, I see... Set doesn't have CollectionElement. Weird. That seems like a bug. You should totally be able to have a list of Sets or a dict of Sets or dare I say it a Set of Sets ... please open an issue! (Ultimately collections should only require Movable not Copyable, but right now they require both, and set isn't currently Copyable.)

Martin Dudek•4mo ago

that is for me at the core of why i often hesitate using List, I dont want to make whatever i want to organize in a List copyable to be able to use List. Right now it seems i have to use UnsafePointer even so i dont like it in particular and I understand Modular discourages it. So to get UnsafePointer out of user Mojo programs, i highly recommend Modular to change the CollectionElement trait as you suggested :mojo:

sora•4mo ago

We will need conditional conformance to make that happen.

Martin Dudek•4mo ago

maybe we find somebody who knows Swift to help us with that 😜

Gaming

Programming

What UnsafePointer can point to and allocate mem for?