Examples of Python code that will never work with Mojo?
Hello, afaik, Mojo is intended to be a Python superset (a sort of Python++), but at the same time, Modular has made it very clear that Mojo won’t be a “Python compiler”, implying that some of the more dynamic Python features won’t work as expected in Mojo (that’s what I understood from some of the Q&A sessions and interviews).
Do you have any specific examples of Python features that will never work with Mojo, not because they aren’t supported yet, but because by design Mojo won’t be able (or willing) to compile them?
For example, redefining intrinsics/builtins? Walking and modifying its own AST and semantics (as some Python-based DSLs do)? Deleting and redefining class methods or attributes? C extensions, C++/Rust bindings?
10 Replies
A really good question, I'm curious too. By looking at Cython, something that they struggled with is the infinitely big integer type by default in Python, which is not performant at all. If you need full compatibility, all standard functions should return this infinitely high int.
But in mojo, using this type would not be performant at all, and doing conversions all the time between integer types too. So this is the current behavior:
And it's very unlikely to change. Use the slow (but safe) int by default everywhere would make mojo fully compatible with python but would make the whole standard library very slow by default.
I'd be curious to know if there are other areas where compatibility will be impractical/impossible. Maybe it's hard to know so much in advance.
Maybe runtime monkeypatching? (It's useless in production, but quite useful when doing unit testing)
good example! I guess the fact that Mojo opted for uppercase Int would allow for using this infinitely larger integer type when lowercase int was encountered, although this wouldn't solve the ambiguity when encountering integer literals
@gabrieldemarmiesse it depends a lot on how compatible you want to be. The major problem is the C interface, which is deeply enmeshed with CPython interpreter internals. You can change these internals (which gives you solid, ~6x improvements in speed over Python without breaking anything outside of C modules), but that breaks standard packages like Pandas/PyTorch/NumPy/anything using C. This is basically what PyPy does.
Alternatively, you can go for full compatibility using something like Cinder or Pyston for a ~2x speed improvement. But because they can't change any of the things that make Python slow (libraries depend on those), that's the most they can really do.
Many current Python programs depend heavily on CPython internals, they're also written with GIL in mind and aren't thread safe.
True but PEP 703 was accepted, and afaik the GIL is more relevant for the C extensions than for python code itself
And that C extension is the issue.
Direct quote from @Chris Lattner: “We haven't built a proper "talk to c python extensions" directly from Mojo subsystem, but when we do, /it/ will have a GIL because c extensions require it, just as you say.”
So it seems that the plan is for Mojo to support C extensions
@Jack Clayton hello Jack, any chances we could get an official approximate answer on this?
Hi @jordiae there is a lot of tricky behavior in Python, I can't give any approximations because I don't know for sure. The bigint behavior that @gabrieldemarmiesse points out above I believe they're close to a solution for. It's a very talented team and they're constantly surprising me, I don't think anything is impossible.
I see, thanks!
Congrats @jordiae, you just advanced to level 2!