[awegio]

> mental models, each with a degree of uncertainty, to some overall business strategy

That actually sounds a lot like model based reinforcement learning. AlphaGo still had an explicit model of a perfect information game, but newer algorithms are capable of learning the rules of games like Minecraft with very little prior information.

The problem is that the real world is still much more complicated and hard to access for machines. However, I wouldn't say it is a fundamentally different problem, but rather a matter of scale. That's part of the reason why AI research spent so much effort on building game playing systems.

[bumby]

There's also some question about the fragility of some of these models, compared to humans [1]. I think there have been some other examples where changing one rule makes their aptitude crash. The implication being they are still hard-rule based, but scale well. But that scale advantage erodes quickly with a rule change because of the lack of contextual understanding.

[1] https://arstechnica.com/information-technology/2022/11/new-g...

[bumby]

I'm not so sure it is a matter of scale. Humans have already lost the scalability race to machines. But I still don't think there is AI developing political policy. Or, if they are, like in the case of setting sentences/bail for criminals/suspects, they show how bad current AI is at it, despite being able to scale more effectively.

'Scaling' IMO is one of the things SWEs harp on a lot, to the extent that they see it as a solution to every problem. This is in part because they recognize that software scales well. When you're a hammer, every problem becomes a nail.