[solarmist]

Not sure I fully understand your comment, but I'm pretty sure the talk addresses exactly that.

The primitives and pre-coded kernels provided by CUDA (it solves for the most common scenarios first and foremost) is what's holding things back and in order to get those algorithms and data structures down to the hardware level you need something flexible that can talk directly to the hardware.

[pjmlp]

C, C++, Fortran, Python JIT from NVidia, plus Haskell, .NET, Java, Futuhark, Julia from third parties, and anything else that can bother to create a backend targeting PTX, NVVM IR, or now cuTile.

The pre-coded kernels help a lot, but you don't have to use them necessarly.