taalhaataahir01022001 Posts

taalhaataahir01022001

•Created by taalhaataahir01022001 on 8/29/2024 in #questions

Dynamic Shapes to MAX GRAPH API

Giving dynamic shape to the graph is giving me error. My code:

fn main() raises:
    var x = Tensor[DType.float32] (1,2,5)
    var y = Tensor[DType.float32] (1,2,5)


    var in_types = List[Type] (TensorType(DType.float32, "a", "b", "c"), TensorType(DType.float32, "a", "b", "c"))
    var graph10 = Graph(in_types=in_types)
    var inputs = List[Symbol] (graph10[0], graph10[1])
    var c = ops.concat(inputs, -1)
    graph10.output(c)
    graph10.verify()
    var session = engine.InferenceSession()
    var concat = session.load(graph10)
 
    var results = concat.execute("input0", x, "input1", y)
    var xd = results.get[DType.float32] ("output0")

    print(x.shape())
    print(y.shape())
    print(xd.shape())

fn main() raises:
    var x = Tensor[DType.float32] (1,2,5)
    var y = Tensor[DType.float32] (1,2,5)


    var in_types = List[Type] (TensorType(DType.float32, "a", "b", "c"), TensorType(DType.float32, "a", "b", "c"))
    var graph10 = Graph(in_types=in_types)
    var inputs = List[Symbol] (graph10[0], graph10[1])
    var c = ops.concat(inputs, -1)
    graph10.output(c)
    graph10.verify()
    var session = engine.InferenceSession()
    var concat = session.load(graph10)
 
    var results = concat.execute("input0", x, "input1", y)
    var xd = results.get[DType.float32] ("output0")

    print(x.shape())
    print(y.shape())
    print(xd.shape())

Error:

Unhandled exception caught during execution: Unsupported dim type: symbolic expression

Unhandled exception caught during execution: Unsupported dim type: symbolic expression

But if I do

var in_types = List[Type] (TensorType(DType.float32, 1, 2, 5), TensorType(DType.float32, 1, 2, 5))

var in_types = List[Type] (TensorType(DType.float32, 1, 2, 5), TensorType(DType.float32, 1, 2, 5))

It works fine. And the issue is only with concat function. I've tried giving dynamic shapes with matmul and it's working fine. Can anyone please guide what I'm doing wrong here?

2 replies

MModular

•Created by taalhaataahir01022001 on 7/26/2024 in #community-showcase

Inference CNN model in mojo (yolo implementation)

I've been working on making a simple inference model in mojo🔥 . A lot of work is still needs to be done and in progress but i made few accomplishments. Wanted to share with the community. I'm new so would love to get feedback on what other things I can try to improve my inference 😄 https://www.linkedin.com/pulse/implementing-yolo-mojo-talha-tahir-an6jf/?trackingId=UDEBRDekTZS%2FQcCEJIBPmg%3D%3D https://github.com/10x-Engineers/Mojo-Yolo

2 replies

MModular

•Created by taalhaataahir01022001 on 6/4/2024 in #questions

Time taken in Inference session

This is my code:

var graph = Graph(in_types=List[Type](TensorType(DType.float32, (N * (H - K + 1) * (W - K + 1)),K * K * C_in), TensorType(DType.float32, K * K * C_in,C_out), TensorType(DType.float32, bias.shape()[0])))
    var out = graph[0] @ graph[1]
    var out1 = out + graph[2]
    var out2 = ops.relu(out1)
    graph.output(out2)
    graph.verify()

    var start_time = now()
    var session = engine.InferenceSession()
    var model = session.load(graph)
    var end_time = now()
    var execution_time : Float32 = (end_time - start_time)
    var execution_time_seconds :  Float32 = execution_time / 1000000000
    print("execution_time_seconds for creating inference session and loadin graph:", execution_time_seconds)

    var out_names = model.get_model_output_names()
    var start_time1 = now()
    var ret = model.execute("input0", A, "input1", wow, "input2", bias)
    var end_time1 = now()
    var execution_time1 : Float32 = (end_time1 - start_time1)
    var execution_time_seconds1 :  Float32 = execution_time1 / 1000000000
    print("execution_time_seconds1 running the inference:", execution_time_seconds1)

var graph = Graph(in_types=List[Type](TensorType(DType.float32, (N * (H - K + 1) * (W - K + 1)),K * K * C_in), TensorType(DType.float32, K * K * C_in,C_out), TensorType(DType.float32, bias.shape()[0])))
    var out = graph[0] @ graph[1]
    var out1 = out + graph[2]
    var out2 = ops.relu(out1)
    graph.output(out2)
    graph.verify()

    var start_time = now()
    var session = engine.InferenceSession()
    var model = session.load(graph)
    var end_time = now()
    var execution_time : Float32 = (end_time - start_time)
    var execution_time_seconds :  Float32 = execution_time / 1000000000
    print("execution_time_seconds for creating inference session and loadin graph:", execution_time_seconds)

    var out_names = model.get_model_output_names()
    var start_time1 = now()
    var ret = model.execute("input0", A, "input1", wow, "input2", bias)
    var end_time1 = now()
    var execution_time1 : Float32 = (end_time1 - start_time1)
    var execution_time_seconds1 :  Float32 = execution_time1 / 1000000000
    print("execution_time_seconds1 running the inference:", execution_time_seconds1)

Output:

execution_time_seconds for creating inference session and loadin graph: 4.304471492767334
execution_time_seconds1 running the inference: 0.00044964000699110329

execution_time_seconds for creating inference session and loadin graph: 4.304471492767334
execution_time_seconds1 running the inference: 0.00044964000699110329

Why creating an inference session and loading the graph into the inference session to create model taking a lot of time i.e., around 4 seconds? Also is there a work around to reduce this time as I have to create different models for my use case. MAX version: max 24.3.0 (9882e19d)

4 replies

MModular

•Created by taalhaataahir01022001 on 5/27/2024 in #questions

LLVM Intrinsics

I'm trying to use llvm.matrix.transpose intrinsic like following:

from sys.intrinsics import llvm_intrinsic
fn main():
    alias intrin: StringLiteral = "llvm.matrix.transpose"

    alias T0 = SIMD[DType.int32, 4]
    alias T1 = Scalar[DType.int32]
    alias T2 = Scalar[DType.int32]
    alias type = SIMD[DType.int32, 4]
    alias has_side_effect: Bool = False

    var a: SIMD[DType.int32, 4] = SIMD[DType.int32, 4] (1, 2, 3, 4)
    var rows: Scalar[DType.int32] = 4
    var cols: Scalar[DType.int32] = 1

    var result = llvm_intrinsic[intrin, type, T0, T1, T2, has_side_effect] (a, rows, cols)
    print(result)

from sys.intrinsics import llvm_intrinsic
fn main():
    alias intrin: StringLiteral = "llvm.matrix.transpose"

    alias T0 = SIMD[DType.int32, 4]
    alias T1 = Scalar[DType.int32]
    alias T2 = Scalar[DType.int32]
    alias type = SIMD[DType.int32, 4]
    alias has_side_effect: Bool = False

    var a: SIMD[DType.int32, 4] = SIMD[DType.int32, 4] (1, 2, 3, 4)
    var rows: Scalar[DType.int32] = 4
    var cols: Scalar[DType.int32] = 1

    var result = llvm_intrinsic[intrin, type, T0, T1, T2, has_side_effect] (a, rows, cols)
    print(result)

But Mojo is crashing:

LLVM ERROR: Cannot select: intrinsic %llvm.matrix.transpose
Please submit a bug report to https://github.com/modularml/mojo/issues and include the crash backtrace along with all the relevant source codes.
Stack dump:
0.    Running pass 'Function Pass Manager' on module '<split-module>'.
1.    Running pass 'X86 DAG->DAG Instruction Selection' on function '@main'
 #0 0x00005f31328dc487 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/home/talha/.modular/pkg/packages.modular.com_max/bin/mojo+0x128c487)
 ...
 ...
mojo crashed!
Please file a bug report.
Aborted (core dumped)

LLVM ERROR: Cannot select: intrinsic %llvm.matrix.transpose
Please submit a bug report to https://github.com/modularml/mojo/issues and include the crash backtrace along with all the relevant source codes.
Stack dump:
0.    Running pass 'Function Pass Manager' on module '<split-module>'.
1.    Running pass 'X86 DAG->DAG Instruction Selection' on function '@main'
 #0 0x00005f31328dc487 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/home/talha/.modular/pkg/packages.modular.com_max/bin/mojo+0x128c487)
 ...
 ...
mojo crashed!
Please file a bug report.
Aborted (core dumped)

My input and output types looks OK as I've followed it from here: https://llvm.org/docs/LangRef.html#llvm-matrix-transpose-intrinsic Also I've tried using other intrinsics and they are working fine. e.g., llvm.vector.reduce.add:

from sys.intrinsics import llvm_intrinsic

fn main():
  alias intrin: StringLiteral = "llvm.vector.reduce.add"
  alias T0 = SIMD[DType.int32, 4]
  alias type = Scalar[DType.int32]
  alias has_side_effect: Bool = False

  var a: SIMD[DType.int32, 4] = SIMD[DType.int32, 4] (1, 2, 3, 40)

  var result = llvm_intrinsic[intrin, type, T0, has_side_effect] (a)

  print(result)

from sys.intrinsics import llvm_intrinsic

fn main():
  alias intrin: StringLiteral = "llvm.vector.reduce.add"
  alias T0 = SIMD[DType.int32, 4]
  alias type = Scalar[DType.int32]
  alias has_side_effect: Bool = False

  var a: SIMD[DType.int32, 4] = SIMD[DType.int32, 4] (1, 2, 3, 40)

  var result = llvm_intrinsic[intrin, type, T0, has_side_effect] (a)

  print(result)

And it outputs 46 correctly. Can anyone guide me what I'm doing wrong while using the llvm.matrix intrinsic?

7 replies

MModular

•Created by taalhaataahir01022001 on 5/21/2024 in #questions

x86 Intrinsics

How to use x86 intrinsics in Mojo? Also I see in the documentation (https://docs.modular.com/mojo/stdlib/sys/intrinsics/llvm_intrinsic) on using llvm_intrinscs. Can someone please share some example on how they've used them. I'm trying to use it but getting error:

from sys.intrinsics import llvm_intrinsic

fn main():
  var intrin: StringLiteral = "llvm.sqrt.f32"
  var type: AnyType
  var has_side_effect: Bool = False

  llvm_intrinsic[intrin, type, has_side_effect] ()

from sys.intrinsics import llvm_intrinsic

fn main():
  var intrin: StringLiteral = "llvm.sqrt.f32"
  var type: AnyType
  var has_side_effect: Bool = False

  llvm_intrinsic[intrin, type, has_side_effect] ()

ERROR:

error: cannot use a dynamic value in call parameter
  llvm_intrinsic[intrin, type, has_side_effect] ()

error: cannot use a dynamic value in call parameter
  llvm_intrinsic[intrin, type, has_side_effect] ()

5 replies

MModular

•Created by taalhaataahir01022001 on 5/21/2024 in #questions

Max Engine Understanding

Can anyone please guide me what's the purpose of MAX engine? This is what I understood: It's some compiler which compiles the model graphs to boost up their performance. But it's not the Mojo compiler. Assuming I write my whole model in Mojo (I implement all the APIs required for the model in Mojo myself). Then Which compiler will be better? If Mojo is going to be faster (assuming I write the best possible code with Mojo 🔥 ) then what's the purpose of MAX Engine 🏎️ ?

9 replies

MModular

•Created by taalhaataahir01022001 on 5/16/2024 in #questions

C/C++ Interpol

https://docs.modular.com/mojo/roadmap#cc-interop When will we have this feature gven in the roadmap? Also right now is their a way to use C++ through mojo?

2 replies

MModular

•Created by taalhaataahir01022001 on 3/26/2024 in #questions

String to float

How can i convert string to float32? e.g.,

var a:String = "-0.01536663"

var a:String = "-0.01536663"

atol is not helping as it's used to convert to integers and not float.

4 replies

MModular

•Created by taalhaataahir01022001 on 3/12/2024 in #questions

GRAPH API MOTensor not working

Facing issues with graph API tutorial: https://docs.modular.com/engine/graph What is this MOTensor?

from max import graph

var m = graph.Module()

var g = m.graph(
    "my_model",
    in_types=MOTensor(DType.float32, 1, 28, 28, 1),
    out_types=MOTensor(DType.float32, 1, 10),
)

from max import graph

var m = graph.Module()

var g = m.graph(
    "my_model",
    in_types=MOTensor(DType.float32, 1, 28, 28, 1),
    out_types=MOTensor(DType.float32, 1, 10),
)

It's not available to me :/

 error: use of unknown declaration 'MOTensor'
    in_types= MOTensor(DType.float32, 1, 28, 28, 1)

 error: use of unknown declaration 'MOTensor'
    in_types= MOTensor(DType.float32, 1, 28, 28, 1)

2 replies

MModular

•Created by taalhaataahir01022001 on 1/18/2024 in #questions

Access Command Line arguments

How Can I access Command line arguments in mojo like we do in python using sys:

import sys
print(sys.argv[1])

import sys
print(sys.argv[1])

Trying the same in mojo:

import sys

fn main() raises:
    print(sys.argv[1])

import sys

fn main() raises:
    print(sys.argv[1])

Yeilds the following error: error: 'argv' expects 0 input parameters, but 1 was specified print(sys.argv[1])

4 replies

MModular

•Created by taalhaataahir01022001 on 1/14/2024 in #questions

Create 1D array of objects

17 replies

MModular

•Created by taalhaataahir01022001 on 1/8/2024 in #questions

How to use f-string in mojo

I want to write a ppm file in mojo using f-string pattern. e.g., thingss like: out.write(f"P3\n{pic.size_X} {pic.size_Y}\n255\n") out.write(f"{clamp(round(r))} {clamp(round(g))} {clamp(round(b))} ") etc. But I can't use f-strings in mojo. Plus if I try to do like this: out.write("P3\n",picture.size_X, picture.size_Y,"\n255\n") It gives me the following error: error: no matching function in call to 'write': out.write("P3\n",picture.size_X, picture.size_Y,"\n255\n") ~~~~~^~~~~~~~~~~~~ /home/talha/Desktop/mojo/jpeg/C-code/Mojo/main.mojo:1:1: note: candidate not viable: callee expects 2 arguments, but 5 were specified Can someone please help me how to write files?

4 replies

MModular

•Created by taalhaataahir01022001 on 1/6/2024 in #questions

convert UInt8 to UInt16

3 replies

MModular

•Created by taalhaataahir01022001 on 1/5/2024 in #questions

How to convert a byte to integer?

4 replies

MModular

•Created by taalhaataahir01022001 on 1/4/2024 in #questions

Dynamic Vector of objects

3 replies

Gaming

Programming