[dayeye2006]

Yes. But some of the algorithms cannot benefit that much from the GPU. In my field -- mathematical optimization, lots of algorithms rely on sparse matrix operations and takes many iterations until convergence.

[modeless]

Would this help? https://jax.readthedocs.io/en/latest/jax.experimental.sparse...

[marmaduke]

Nope it's super slow for large sparse matrices. It's even faster to use generic scatter/gather to implement some, instead of that built in thing.