Trying to understand Parallelize
I have some code which I would like to speed up and cannot vectorize since it uses custom types. I am trying to use parallelize to split up the compute and am running in a Docker container on Intel Mac, Mojo 0.7.0. I have tried many combinations of parallelize[func](jobs, workers) but the best improvement I can get is about 2.5x. Why can I not do any better?
1 Reply
Firstly, id need to know specific implementations but you likely COULD vectorize it with some small tweaks. Secondly, there are two downsides to parrallelizing (That vectorizing doesnt suffer from): The significant overhead of staging new jobs and waiting for all to finish, as well as breaking up the work for the jobs, and the ammount of cores available to Mojo at a given time. I would recommend looking into your code and finding way to vectorize it, you are welcome to DM me if you would lieks ome help