[fermenflo]

I agree, a lot of the code could be improved. But some of what you mentioned is fairly standard. Like "Gaussian Error Linear Units being GELU, w/b for weights/biases, etc...

[jsinai]

Not sure how standard that is ...

[steve_musk]

It’s very standard ML abbreviations.