Tried the new Max custom_ops examples with my RTX 3050 and using CPU
When I run the custom_ops examples in nightly branch I realized that they were slower than in the demo, and then I checked that
accelerator_count() was returning 0 even if I have an Nvidia GPU:
12 Replies
Try a driver update, those drivers are about a year old at this point.
It works fine for me on my 4090.
We should support Ampere and newer GPUs (with the possible exception of the Jetson Orin), and the RTX 3050 should fall in the
sm_80 CUDA capabilities that we support. If the GPU was found, but was an older CUDA architecture than we support you'd get a different error message. Seems like it's somehow not finding the GPU at all.Congrats @Brad Larson, you just advanced to level 2!
Would the 535 drivers be too old?
I think that's pre-nvidia-open.
I will say that the A10, A100, L4, L40 are our initial officially supported NVIDIA GPUs. Other Ampere and newer GPUs may work, but they're "use at your own risk" in terms of support right now.
Is there a chance that could be written down as a support matrix for MAX? An "official support" list, and then a "should work, community support only" list?
Some other information like driver versions would be nice.
We've updated the system requirements (Linux tab) with the officially supported GPUs: https://docs.modular.com/max/faq/#system-requirements . We can open up a Discourse thread for more informal discussions about GPU support.
MAX FAQ | Modular Docs
Answers to various questions about MAX.
My main concern is people expecting to jump in with their GPU and start replacing pytorch, so providing a technical reason for the "older than Ampere won't work" may help. If I'm taking a wild guess is has to do with tensor core features/changes.
I started a Discourse thread to at least provide a place to list officially-supported hardware and discuss anything beyond that which people have found to work: https://forum.modular.com/t/nvidia-hardware-support-in-max-24-6/340
Modular
NVIDIA hardware support in MAX 24.6
With the support for NVIDIA GPUs launching in MAX 24.6, a number of people have already asked what hardware is compatible with MAX in this release. This thread is intended to be a reference for our officially-supported hardware, as well as a place for community discussions about hardware compatibility. Our officially-supported NVIDIA GPUs are l...
Thank you, I'll bookmark that to point people to in the future.
@Darkmatter thanks so much for your tip! It worked perfectly after upgrading the nvidia drivers:
I updated the forum thread with that.