Modular•2mo ago

to_numpy with Mojo 24.5

I try to convert the to_numpy method from the Mojo 24.4 based KMeans tutorial https://www.modular.com/blog/fast-k-means-clustering-in-mojo-guide-to-porting-python-to-mojo-for-accelerated-k-means-clustering to Mojo 24.5 but get the following error:

error: 'pop.index_to_pointer' op MLIR verification error: unregistered operation 'pop.index_to_pointer' found in dialect ('pop') that does not allow unknown operations

error: 'pop.index_to_pointer' op MLIR verification error: unregistered operation 'pop.index_to_pointer' found in dialect ('pop') that does not allow unknown operations

The orginal method is implemented as follows:

fn to_numpy(self) raises -> PythonObject:
        var np = Python.import_module("numpy")
        var np_arr = np.zeros((self.rows,self.cols))
        var npArrayPtr = DTypePointer[dtype](
        __mlir_op.`pop.index_to_pointer`[
            _type = __mlir_type[`!kgen.pointer<scalar<`, dtype.value, `>>`]
        ](
            SIMD[DType.index,1](np_arr.__array_interface__['data'][0].__index__()).value
        )
    )
    memcpy(npArrayPtr, self._matPtr, len(self))
    return np_arr ^

fn to_numpy(self) raises -> PythonObject:
        var np = Python.import_module("numpy")
        var np_arr = np.zeros((self.rows,self.cols))
        var npArrayPtr = DTypePointer[dtype](
        __mlir_op.`pop.index_to_pointer`[
            _type = __mlir_type[`!kgen.pointer<scalar<`, dtype.value, `>>`]
        ](
            SIMD[DType.index,1](np_arr.__array_interface__['data'][0].__index__()).value
        )
    )
    memcpy(npArrayPtr, self._matPtr, len(self))
    return np_arr ^

I replaced DTypePointer[dtype] with UnsafePointer[Scalar[dtype]] Any advice highly appreciated 😉

Modular: Fast⚡k-means clustering in Mojo🔥: a guide to porting Pytho...

There are several clustering algorithms, but k-means — the algorithm we're going to implement from scratch in Python and Mojo🔥 in this blog post — is one of the most popular due to its simplicity and ease of implementation.

6 Replies

TilliFe•2mo ago

I think I have been there too. Does the following example from the MAX repo help as a reference? https://github.com/modularml/max/blob/434daac5b52226d8c7ea024d08df127af5fce9dd/examples/serve/openclip-mojo-onnx/python_utils.mojo#L61

GitHub

max/examples/serve/openclip-mojo-onnx/python_utils.mojo at 434daac5...

A collection of sample programs, notebooks, and tools which highlight the power of the MAX Platform - modularml/max

Darin Simmons•2mo ago

Just to be clear, you updated DTypePointer to UnsafePointer (due to deprecation) and got the above error? @Shashank P

DobyDabaDu•2mo ago

You can check mine, it works: https://github.com/yetalit/Mojmelo/blob/main/mojmelo/utils/Matrix.mojo

GitHub

Mojmelo/mojmelo/utils/Matrix.mojo at main · yetalit/Mojmelo

Machine Learning algorithms in pure Mojo 🔥. Contribute to yetalit/Mojmelo development by creating an account on GitHub.

DobyDabaDu•2mo ago

875 th line

Martin DudekOP•2mo ago

thanks @TilliFe , @Darin Simmons and @DobyDabaDu - quite some other 24.4 -> 24.5 issues in the project i am working on so I cant check what works right now but will post here once it's done .... Thanks again 🙏 I ended up using @DobyDabaDu implementation which works like a charm. Thanks again for the all the feedback :mojo:

DobyDabaDu•2mo ago

Glad it worked❤️

Gaming

Programming

to_numpy with Mojo 24.5