Modular•9mo ago

Difference between Float32 and DType.float32

I don’t have a ton of experience with python or AI/ML and so trying to figure out this basic questions in the comments around the difference / similarity between Float32 and DType.float32 more context in this snippet:

let dtp = DTypePointer[DType.float32]().alloc(10)
let p = Pointer[Float32].alloc(10)
for i in range(10):
    let fi = Float32(i)
    p[i] = fi

    # is a Dtype.float32 same as a Float32 in terms of memory layout?
    # what is the difference between DType.float32 and Float32?
    dtp[i] = fi

    # how do you create a DType.float32?
    # let fi2 = DType[DType.float32](i)
    # let dtp[i] = fi2

let dtp = DTypePointer[DType.float32]().alloc(10)
let p = Pointer[Float32].alloc(10)
for i in range(10):
    let fi = Float32(i)
    p[i] = fi

    # is a Dtype.float32 same as a Float32 in terms of memory layout?
    # what is the difference between DType.float32 and Float32?
    dtp[i] = fi

    # how do you create a DType.float32?
    # let fi2 = DType[DType.float32](i)
    # let dtp[i] = fi2

5 Replies

Arthur Evans•9mo ago

TL;DR: DType describes a data format in the abstract. Types like DTypePointer, SIMD, and Buffer use DTypes to describe the data they store. Float32 is a concrete type which you can instantiate. These aren't really related to AI/ML or Python.Float32 is a type. You can instantiate a Float32, like you did in your code (Float32(i)). DType is a structure that provides various utilities for working with data types, including enumerated values (enums) describing various data types, like float32. So DType.float32 is a value that describes a type. It's used in a variety of generic types like DTypePointer and SIMD to describe the type of data being stored. You can't create a DType.float32, but you can create a Float32(which is a scalar value of that dtype) or a Buffer or SIMD vector made up of DType.float32 values. I'm going to take a short tangent here. I swear it's related. I don't know if you've touched on SIMD yet--it's another type that tends to confuse folks, and this part is kind of related to AI/ML. SIMD (single instruction, multiple data) is a technique for parallelizing data operations when you want to do a lot of similar operations. A Mojo SIMD type is a small, fixed-sized vector--or array--of numeric data types. These are used heavily in low-level AI/ML programming, so they're built right in to the Mojo standard library. A SIMD[DType.float32, 8] is a vector of 8 Float32 values. You can do something fancy like take the square root of all of the values with a single operation:

var s = SIMD[DType.float32, 8](1, 2, 3, 4, 5, 6, 7, 8)
var s2 = math.sqrt(s)

var s = SIMD[DType.float32, 8](1, 2, 3, 4, 5, 6, 7, 8)
var s2 = math.sqrt(s)

With SIMD, doing 8 operations at once takes about the same time as doing one—so you can see why this is desirable when you're crunching a lot of numbers. OK, so I told you this was related, and it is. What exactly is that Float32 type. you're using? It's actually an alias for SIMD[DType.float32, 1]. That's right, it's a SIMD vector holding one 32-bit floating point number. When you create a DTypePointer[DType.float32], it's basically the same as creating pointer to an array of Float32 scalar values.

Mohamed Mabrouk•9mo ago

@Arthur Evans following up on the question, as I understadn Float32 is a type defined in the standard library. is Dtype.float32 is also part of the standard library or it is implemented in the compiler refereing to MLIR values? also can you defined more DTypes in Mojo using the __mlir* syntax? lets say Dtype.bfloat16, or Dtype.float5 (or any custom width numeric).

Arthur Evans•9mo ago

They're both defined in the standard library. There's currently no mechanism to add new DTypes (there is already one for bfloat16, but I don't know how complete the support is). You can find the list of current DTypes here: https://docs.modular.com/mojo/stdlib/builtin/dtype.html#dtype Theoretically you can write a new type, as described in Low-Level IR with Mojo: https://docs.modular.com/mojo/notebooks/BoolMLIR.html ... but this is a really advanced topic that we don't have any documentation for at the moment.

Modular Docs - Low-level IR in Mojo

Learn how to use low-level primitives to define your own boolean type in Mojo.

Modular Docs - dtype

Module

Mohamed Mabrouk•9mo ago

Thnx

clarkezone•9mo ago

@Arthur Evans that is super helpful! Would be great if the documentation explained the above (i'm contemplating a PR based on your explanation ;-)). I also can't wait for debugger support to be able to inspect memory layout.

Gaming

Programming

Difference between Float32 and DType.float32