Will Mojo Metaprogramming support Lisp/Julia Style Macros?
As Paul Graham is describing in http://www.paulgraham.com/avg.html Lisp let you abstract things with macros which can’t still be easily abstracted in current popular languages. If this would be realized in Mojo, we could write less Code and be even more productive.
30 Replies
Then app developers can create their own syntactic sugar with macros if the mojo team don’t have the time to implement it as far I understood
See https://docs.modular.com/mojo/programming-manual.html#parameterization-compile-time-metaprogramming
Modular Docs - Mojo🔥 programming manual
A tour of major Mojo language features with code examples.
It states it supports python like metaprogramming at compile time. Since lisp/racket/Julia metaprogramming are not the same as python metaprogramming, I am asking if they will support this in the future
I mean tell gpt-4 for example Use Julia metaprogramming to make singleton design pattern part of the language syntax and compare to regular code. Instead of writing a singleton boilerplate for every new class you have now a new singleton keyword, so a one liner
I think this would be greAt for mojo since the compiler team can focus on the important core stuff and keep the core system simpler and syntactic sugar and language extensions like async await can be implemented easily by normal developers and shared
GitHub
[Feature Request] Syntactic Macros and First-Class Metaprogramming ...
Review Mojo's priorities I have read the roadmap and priorities and I believe this request falls within the priorities. What is your request? First-class metaprogramming with syntactic macros (...
I agree with @Chris Lattner that language features should be motivated by problems but in this case in my opinion it is already motivated by the biggest programmer problem. Managing complexity. For example we agree that oop helps to manage complexity and languages like c could emulate oop. So I assume if c would support Julia like metaprogramming a library with macros could add oop syntax to c to manage complexity without the longer process to add language features by the language designers and them having an agreement first and without increasing the complexity of the core language, opposite to c++ and all it’s features. New Language constructs could first be fast empirical tested through user libraries and if validated be added to the core language. Maybe it would also help in the language implementation because more patterns can be faster abstracted to manage complexity. Sorry if I said something wrong because I am not an expert.
Congrats @david, you just advanced to level 1!
Ok only 0.4 % of Julia stdlibs Code are macros. So it is probably something which should not be used so often anyways
Just my opinion but there's no need to have an exact or similar metaprogramming model as Julia or Lisp, just the same methodology. A lot of those cases are solved at compile time, hence the focus of Mojo listed in the documentation mentioned above on this aspect. The benefits of compile-time are performance-related.
We'll have to explore this over time. I'm interested in allowing library authors to make things that look like builtin control flow statements ("for" is not enough, let's get "parallel_for") and supporting things like patterns can only (afaik) realistically be done with a macro like system.
That said, as you probably know, there is a huge design space for macro systems. We'll need to see if and what actually makes sense for various usecases.
This is all to say "yeah we'll likely do it, but want to make sure it is done the right way, and maximal power isnt' really the goal"
Afaik in case of abstracting patterns of behaviors there is an overlap between macros and higher order functions. So you could have a higher order function implementing the for loop and parallelize logic and inject a function as an argument, which is executed in parallel for every element. Macros are more general and act similar to code generators. In theory a programmer could design a new language for a Problem domain in mojo and a non technical domain expert could „code“ apps with this, reducing the technical barriers for businesses. Here a code example for circuit analysis in Julia c = Parallel(Resistor(5), Serial(Capacitor(1), Inductor(3))) with macros becomes c = (at)circuit R(5) // (C(1) —> I(3)). Because Julia treats internally Code as data, these code transformations are easier and natural to code.
Would you consider a macro system where you could have first order logic similar syntax in mojo too powerful? It would be cool if you could just take the paper math and that would already be the program to a big extent
I mean Mojo tries to solve the N world problem. Having a good Macro system solves it partly since you still have to map from the math or business world to the Imperative programming world
I agree that OOP helps to manage complexity, but do not focus too much on Clean Code or OOP, because then you suffer a performance hit in your code a lot. For performance rich code, focus more on data-oriented design like it is done in game development and put OOP onto the shell.
For more reference, have a search for Casey Muratori or Richard Fabian.
We haven't designed a macro system, so I can't speak to specific design points. We'll have to see as other more fundamental pieces of the type system come together.
Actually I found a simpler use case which could save much time, porting Python libraries to Mojo. For example in the matrix multiplication example in the docs the optimized Mojo implementation looks like # Use the above tile function to perform tiled matmul.
fn matmul_tiled_parallelized(C: Matrix, A: Matrix, B: Matrix):
@parameter
fn calc_row(m: Int):
@parameter
fn calc_tile[tile_x: Int, tile_y: Int](x: Int, y: Int):
for k in range(y, y + tile_y):
@parameter
fn dot[nelts : Int,](n : Int):
C.store[nelts](m,n + x, C.loadnelts + A[m,k] * B.loadnelts)
vectorizenelts, dot
# We hardcode the tile factor to be 4.
alias tile_size = 4
tile[calc_tile, nelts * tile_size, tile_size](A.cols, C.cols)
parallelize[calc_row](C.rows, C.rows)
It would be extraordinary, if you could have the performance boost, while maintaining the readability of the pseudocode or naive implementation like
Note that C, A, and B have types.
fn matmul_naive(C: Matrix, A: Matrix, B: Matrix):
@parallelize for m in range(C.rows):
@tile for k in range(A.cols):
@vectorize for n in range(C.cols):
C[m, n] += A[m, k] * B[k, n]
Maybe the macros could generate even more optimized code, which is harder to read and engineer.
This would benefit in my opinion the also the adoption of Mojo, because Python Libraries maintainer have to rewrite less to get the best out of Mojo.
With this approach you could also probably keep the debugging experience of non macro code. When you run in debug mode you would basically ignore the macros. You would debug as regular code. In release Mode the Macro is expanded and gives the performance boost. This approach would also benefit compilation speed, since most compilation happens in debug mode.
def matmul_untyped(C, A, B):
@parallelized
for m in range(C.rows):
@tiled
for k in range(A.cols):
@vectorized
for n in range(C.cols):
C[m, n] += A[m, k] * B[k, n]
This is more consistent with the syntax like the @parameter
It sounds like you're reinventing LoopVec/PythonSyntax.jl here 😆
Relatedly, I'd certainly love it if there were some way to get compatibility with Julia out of Mojo, so people could migrate from Julia to Mojo easily as well. There's tons of open-source code written for performance in Julia that could be ported over easily. (Unlike Python, where most code relies heavily on CPython internals that slow it down.)
@Maximum Limelihood Estimator I think, there is nothing particularly Julia here, as most languages/compiler toolchain that offers, say, manual loop vectorization, requires you to use some form of pragma/annotation/decorator.
Though I really like/want to use the decorator form of
vectorize etc, especially when we make for statement a library instead of a builtin, I do wonder if they as capable as the function form.The reason I say it's very Julia is because of the use of macros/syntax tree representations to rewrite this; it's definitely in the spirit of Lisps or Julia
pragmas/annotations/decorators are all different from (much less general than) syntactic macros. Pragmas and annotations are just instructions to the compiler that have to be special-cased and can't be extended by the user, because they're part of the compiler itself. Decorators are a bit more flexible but still only let you wrap a function (you can't do anything that you couldn't do with a higher-order function).
A macro takes code and represents it as a syntax tree of expressions, then rewrites the syntax tree into something else. This basically means you get to write your own compiler extensions to add new syntax, by translating one piece of syntax to another. With syntactic macros you can introduce any new syntax you'd like, but it stays limited to a local subset of your code where you applied the macro
I use the word decorator in an abstract way, that is, anything that looks like a decorator, not necessarily a python decorator. My point is, in the code snippet we were only given a decorated for loop, and we just couldn't know the technology used to implement it. For instance, the
unroll we have now, is not implemented as a macro.
That said, I don't disagree with you, for what you said are mostly trivially true (I don’t mean it in a bad way). And I agree syntactic macro is a nice to have.Ahh, then yeah, it's not exclusive to Julia; Lisp, Dylan, and Nim have it too. But syntactic macros definitely aren't that common!
@Maximum Limelihood Estimator There is also design space covered by ppx for ocaml or template haskell. They are not exactly macro system, but they do drew ideas from lisp, and I think they fit a non-lisp language (such as mojo) better. And apologies if my tone was off in our last exchange.
Oh gosh don't worry, you're totally fine :)
I don't know much about OCaml or Haskell, but that sounds interesting. I'll make sure to check it out!
Congrats @Maximum Limelihood Estimator, you just advanced to level 1!
Rhombus is also very interesting. I think there are many ideas we can steal. https://2023.splashcon.org/details/splash-2023-oopsla/52/Rhombus-A-New-Spin-on-Macros-without-All-the-Parentheses
Rhombus: A New Spin on Macros without All the Parentheses (SPLASH 2...
The ACM SIGPLAN International Conference on Systems, Programming, Languages and Applications: Software for Humanity (SPLASH) embraces all aspects of software construction and delivery, to make it the premier conference on the applications of programming languages - at the intersection of programming languages and software engineering.
We welcom...
i think mojo is supporting template metaprogramming
If I remember correctly, Chris explicitly mentioned mojo meta programming won’t be like template in cxx. And I’d say, thankfully it’s noting like that.
From what I can tell, it's still templates, just with a different syntax that's built into the language (Like Zig). But non-template metaprogramming hasn't been ruled out
If you take the definition of temp to be “templates are used by a compiler to generate temporary source code, which is merged by the compiler with the rest of the source code and then compiled” then lisp macros (in contrast to language like Julia) are more template than syntactic macro as there is really no separate ast type, and macro expend to code and not some reified representation of code. All that is to say the terminology is very complex/confusing and probably not very helpful.
I don’t think people will call python decorators templates. And the reason is like very python specific: python is a statement based language, and every statement is runnable (not declaration, class keyword actually creates a class and you can put it in a for loop). In the case of our compiled python, mojo, a phase distinction is introduced. But if one takes the view of compiling as some form of partial evaluation, then what we have now is pretty much still (at least achievable with) python style (higher order function++) metaprogramming.
Congrats @sora, you just advanced to level 6!
The difference between the two is that Julia or Lisp let you capture and modify the AST directly. You're right that Lisp doesn't have a separate type to represent ASTs, but that's because in Lisp, every "statement" is a list, and the way you specify a program is by writing out the AST as a list; in other words, in Lisp, everything is an AST. (And all other data types are represented in terms of ASTs).
Julia, Nim, and Dylan make the AST a bit less fundamental to the language, because they're all statement-based, but as the first step of compilation they transform the statements into an AST before handing the AST off to any macros for rewriting.
Congrats @Maximum Limelihood Estimator, you just advanced to level 2!
By everything is an AST, I think you meant every thing is (or at least representable as) a s-expression, or simply, everything is a tree, right? Otherwise, it sounds a bit backward. Also, isn't julia expression based (mostly everything evals to some value)?