teïlo
MModular
•Created by teïlo on 10/16/2024 in #community-showcase
Operatio
Hello Mojo Community !
I'm happy to share Operatio, a tool that uses Mojo to perform task arithmetic (task vector operations) on language models.
Initially, I attempted to implement this in Python using MAX, but encountered some challenges. This led me to explore Mojo! (I've kept the Python version as pyoperatio for reference.)
As a newcomer to many of these concepts, I've been on a steep learning curve with tensors, SIMD, and Mojo itself. (I am still not sure I do understand all of it, to be honest). And after some days with o1 and claude, I managed to make something.
I'm certain there's room for improvement, so please share your suggestions!
As I was trying to optimize some parts, I've used
vectorize for the extract and transform operations, replacing my initial for-loop approach. Interestingly, I haven't observed significant time improvements. o1 told me that it could be due to vectorization overhead or the relatively small scale of the models I've tested with (1B). I'm curious about this - perhaps I didn't implement it optimally for the task?
This project has been a lot of fun, and I hope to come back with updates.
https://github.com/teilomillet/operatio6 replies