[Jack000]

The thing is, it's not gibberish. A sufficiently small language model can be understood by humans: https://twitter.com/karpathy/status/1645115622517542913

The explanation is perfectly sensical, just too complex for humans to understand as the model scales up.

The thing you're looking for - a reductive explanation of the weights of a ANN that's easy to fit in your head, does not exist. If it were simple enough to satisfy your demands, it wouldn't work at all.

[zelphirkalt]

Yet, when a master player makes a decision what move to play, they often have concrete reasons for it, that they discuss in after game analysis. They evaluate some advantage or chances higher than others or some risks greater than others and calculate specific sequences ahead to be sure to solve a subproblem correctly and base their decision on that.