franchesoni Posts - Answer Overflow

franchesoni

•Created by franchesoni on 10/23/2024 in #questions

dict containing sets?

I'm having issuesto have a dict whose values are sets first, Set doesn't implement copyinit so I had to wrap it but now I get that rag[current_label] is a "function that might raise in a context that cannot" what's the simplest way to build a dict whose values are sets? can I operate on the sets? here's the code if needed

struct CopyableSet[T: KeyElement](CollectionElement):  # we need this because set is not copyable :S
    var set: Set[T]

    fn __init__(inout self: Self):
        self.set = Set[T]()

    fn __copyinit__(inout self: Self, existing: Self):
        self.set = Set[T]()
        self.set._data = existing.set._data

    fn __moveinit__(inout self: Self, owned other: Self):
        self.set = other.set^

fn create_rag(indices: FlatTensor[DType.uint16]) -> Dict[Int, CopyableSet[Int]]:
    neighbor_offsets = List[List[Int]](
        List(1, 0), List(-1, 0), List(0, 1), List(0, -1)
    )
    rag = Dict[Int, CopyableSet[Int]]()
    max_label, _ = max[DType.uint16](indices, 0, len(indices.buffer))
    for label in range(1, max_label + 1):
        rag[label] = CopyableSet[Int]()

    for row in range(S):
        for col in range(S):
            current_label = indices.buffer[rowcol_to_ind(row, col)].__int__()
            if current_label == 0:
                print('tremendous mistake!')
               
            for i in range(len(neighbor_offsets)):
                nrow = row + neighbor_offsets[i][0]
                ncol = col + neighbor_offsets[i][1]
                if 0 <= nrow and nrow < S and 0 <= ncol and ncol < S:
                    neighbor_label = indices.buffer[rowcol_to_ind(nrow, ncol)]
                    if neighbor_label != current_label:
                        aux = rag[current_label]  # this line fails
                        aux.set.add(neighbor_label.__int__())
                        rag[current_label] = aux
    return rag

struct CopyableSet[T: KeyElement](CollectionElement):  # we need this because set is not copyable :S
    var set: Set[T]

    fn __init__(inout self: Self):
        self.set = Set[T]()

    fn __copyinit__(inout self: Self, existing: Self):
        self.set = Set[T]()
        self.set._data = existing.set._data

    fn __moveinit__(inout self: Self, owned other: Self):
        self.set = other.set^

fn create_rag(indices: FlatTensor[DType.uint16]) -> Dict[Int, CopyableSet[Int]]:
    neighbor_offsets = List[List[Int]](
        List(1, 0), List(-1, 0), List(0, 1), List(0, -1)
    )
    rag = Dict[Int, CopyableSet[Int]]()
    max_label, _ = max[DType.uint16](indices, 0, len(indices.buffer))
    for label in range(1, max_label + 1):
        rag[label] = CopyableSet[Int]()

    for row in range(S):
        for col in range(S):
            current_label = indices.buffer[rowcol_to_ind(row, col)].__int__()
            if current_label == 0:
                print('tremendous mistake!')
               
            for i in range(len(neighbor_offsets)):
                nrow = row + neighbor_offsets[i][0]
                ncol = col + neighbor_offsets[i][1]
                if 0 <= nrow and nrow < S and 0 <= ncol and ncol < S:
                    neighbor_label = indices.buffer[rowcol_to_ind(nrow, ncol)]
                    if neighbor_label != current_label:
                        aux = rag[current_label]  # this line fails
                        aux.set.add(neighbor_label.__int__())
                        rag[current_label] = aux
    return rag

4 replies

MModular

•Created by franchesoni on 10/17/2024 in #questions

mojo compile is non-deterministic ? (video)

I only call mojo run main.mojo again and again, and - it breaks - it gives a result - it gives another result I can't see where the randomness is coming from, it is certainly not intended

6 replies

MModular

•Created by franchesoni on 10/17/2024 in #questions

print breaks code

the following isn't catched by the linter but breaks, what am I doing wrong?

from python import Python, PythonObject
from memory import memcpy

alias S: Int = 256
alias S2: Int = S*S
alias CRASH = True

def main():
    np = Python.import_module('numpy')
    pylogits = np.ones((130, 256, 256))
    total_size = Int(pylogits.size)

    # now we have the data in memory
    data = UnsafePointer[Scalar[DType.float32]].alloc(total_size)
    memcpy(data, pylogits.__array_interface__['data'][0].unsafe_get_as_pointer[DType.float32](), total_size)

    # mojomain(data, total_size)
    var masks = UnsafePointer[Scalar[DType.bool]].alloc(total_size)
    for channel in range(total_size / S2):
        offset = S2 * channel
        for ind in range(S2):
            masks[ind + offset] = True if data[ind+offset] > 0.5 else False
        if CRASH:
            print(masks[offset + S])

    print('great!')  # never get here

from python import Python, PythonObject
from memory import memcpy

alias S: Int = 256
alias S2: Int = S*S
alias CRASH = True

def main():
    np = Python.import_module('numpy')
    pylogits = np.ones((130, 256, 256))
    total_size = Int(pylogits.size)

    # now we have the data in memory
    data = UnsafePointer[Scalar[DType.float32]].alloc(total_size)
    memcpy(data, pylogits.__array_interface__['data'][0].unsafe_get_as_pointer[DType.float32](), total_size)

    # mojomain(data, total_size)
    var masks = UnsafePointer[Scalar[DType.bool]].alloc(total_size)
    for channel in range(total_size / S2):
        offset = S2 * channel
        for ind in range(S2):
            masks[ind + offset] = True if data[ind+offset] > 0.5 else False
        if CRASH:
            print(masks[offset + S])

    print('great!')  # never get here

12 replies

MModular

•Created by franchesoni on 10/17/2024 in #questions

List of SIMD bug

The following simple code doesn't work, is this because of memory? I find the documentation is lacking, it only says SIMD are restricted to powers of 2, which I compy with...

alias S: Int = 256
alias S2: Int = S*S
alias SquareMatrix = SIMD[size=S2]

def main():
    var sqms = List[SquareMatrix[DType.float32]]()
    for ind in range(2):  # using 1 instead of 2 gives no issues
        sqm = SquareMatrix[DType.float32]()
        sqms.append(sqm)
    print('success!')  # we never get here :(

alias S: Int = 256
alias S2: Int = S*S
alias SquareMatrix = SIMD[size=S2]

def main():
    var sqms = List[SquareMatrix[DType.float32]]()
    for ind in range(2):  # using 1 instead of 2 gives no issues
        sqm = SquareMatrix[DType.float32]()
        sqms.append(sqm)
    print('success!')  # we never get here :(

77 replies

MModular

•Created by franchesoni on 10/2/2024 in #questions

how to use Buffer?

it seems we're still in the dark regarding arrays. Mojo has efficient implementations of max, sum, etc. over Buffers but creating one is a pain. For instance this code breaks on the print, why?

    np = Python.import_module('numpy')
    logits = np.load('logits.npy')  # (C, H, W)

    # create data buffer
    var channels: Int = logits.shape[0]
    var height: Int = logits.shape[1]
    var width: Int = logits.shape[2]
    pointer_to_data = UnsafePointer[Float32].alloc(channels * height * width)
    pointer_to_data.init_pointee_copy(0)  # is this needed? idk, doesn't hurt
    tensor = NDBuffer[DType.float32, 3](pointer_to_data, DimList(channels, height, width))
    var ftensor = tensor.flatten()  # I need to create the NDBuffer first as I can't create buffer with a dynamic size
    var np_pointer = logits.__array_interface__['data'][0].unsafe_get_as_pointer[DType.float32]()  # from some example

    start = time.perf_counter()  # copy np data to tensor data
    for channel in range(channels):
        for row in range(height):
            for col in range(width):
                var posind = channel * height * width + row * width + col
                ftensor[posind] = np_pointer[posind]
    print("Time elapsed: ", time.perf_counter() - start)
    print(ftensor[0])  # breaks here

    np = Python.import_module('numpy')
    logits = np.load('logits.npy')  # (C, H, W)

    # create data buffer
    var channels: Int = logits.shape[0]
    var height: Int = logits.shape[1]
    var width: Int = logits.shape[2]
    pointer_to_data = UnsafePointer[Float32].alloc(channels * height * width)
    pointer_to_data.init_pointee_copy(0)  # is this needed? idk, doesn't hurt
    tensor = NDBuffer[DType.float32, 3](pointer_to_data, DimList(channels, height, width))
    var ftensor = tensor.flatten()  # I need to create the NDBuffer first as I can't create buffer with a dynamic size
    var np_pointer = logits.__array_interface__['data'][0].unsafe_get_as_pointer[DType.float32]()  # from some example

    start = time.perf_counter()  # copy np data to tensor data
    for channel in range(channels):
        for row in range(height):
            for col in range(width):
                var posind = channel * height * width + row * width + col
                ftensor[posind] = np_pointer[posind]
    print("Time elapsed: ", time.perf_counter() - start)
    print(ftensor[0])  # breaks here

I have already spent quite a few hours trying to figure out why creating a Buffer is so much trouble. Also, slicing is hard. I have found no good resources on this, kapa tells me there is not enough info in the docs 😦

2 replies

MModular

•Created by franchesoni on 2/19/2024 in #questions

using mojo in python (not python in mojo)

sometimes I need to make only one function really really fast, but I don't want to stop using python because it is so convenient. How do I handle this case? This goes probably against mojo philosophy because creates a two-world problem (python on top of mojo) but I want it so hard

2 replies

Gaming

Programming