[mamp]

It turns out to accurately predict the next word requires huge amounts of implicit contextual knowledge i.e. understanding about the world.

Next word prediction was the trigger, the optimisation for the task results in a broad (but currently unreliable) model of the world.