[bollu]

You're right on one count: no library implements forward mode. Hence, you're correct that no autodiff library (including pytorch) implement autodiff this way.

However, *you're wrong* that forward mode cannot be written in terms of dual numbers. The point is that the addition and multiplication operation for dual numbers correspond exactly to rules of the derivative of addition and the derivative of the product.

[fpgamlirfanboy]

i didn't say it cannot be, i said there's a natural reason it's not: the same reason the rest of analysis isn't.

> You're right on one count: no library implements forward mode

there are plenty of CFD type libraries that implement forward mode. they also do not use what i'm calling "magical" dual numbers i.e. these nilpotent ring elements.