KANs in Mojo - second attempt
This week I finally found time to dive into Kolmogorov–Arnold Networks again and give a Mojo implementation another shot.
https://github.com/dorjeduck/kamo
I decided to scrap my original source code and port A from-scratch implementation of Kolmogorov-Arnold Networks (KAN)…and MLP to Mojo.
Turns out, this base made it much easier for me to grasp the topic, especially after all the confusion around the many derivatives involved in KANs. 😉
Right now, the implementation is just a learning project for me, and it doesn't have any particular wider benefits for the community I am afraid. I might improve on it to make it more competitive within the ever-growing ocean of KAN implementations, but it's not a top priority for me at the moment ...
GitHub
GitHub - dorjeduck/kamo: Kolmogorov-Arnold Networks in Mojo
Kolmogorov-Arnold Networks in Mojo. Contribute to dorjeduck/kamo development by creating an account on GitHub.
Lorenzo Maggi
ML without tears
A from-scratch implementation of Kolmogorov-Arnold Networks (KAN)…a...
Kolmogorov-Arnold networks (KAN) are generating significant interest in the AI community due to their potential for accuracy and interpretability. We implement KAN (and MLPs, incidentally) from scr…
2 Replies
I have added Gaussian radial basis functions as edge function option to the repo, inspired by https://github.com/ZiyaoLi/fast-kan
They are nearly twice as fast to compute compared to the standard BSpline basis functions and seem to work well with the toy examples i am testing with right now.
Overall, it's becoming clear to me that KANs, while conceptually beautiful, are not simple plug-and-play networks. Significant refinement of my network implementation (sparsification, grid extension etc) seems necessary to make it suitable for more real-world applications. Time will tell 😉
GitHub
GitHub - ZiyaoLi/fast-kan: FastKAN: Very Fast Implementation of Kol...
FastKAN: Very Fast Implementation of Kolmogorov-Arnold Networks (KAN) - ZiyaoLi/fast-kan
I’m in love