[lorenzohess]

From the Table, all models are overwhelmingly Regulatory, with smollm2:1.7b being the only one that's majority Libertarian.

All models are overwhelmingly Progressive, with claude-sonnet-4-5-20250929 and grok-4-fast-non-reasoning being the only ones that are majority Conservative.

While there's a bit more balance across other categories (by inspection) it seems like LLMs reflect today's polzarization?

It would be interesting to have statistics about the results which reflect polarization. Perhaps we could put each LLM on the political compass? Also weight the result by the compliance (% results that followed prompt instructions).

[miroljub]

> While there's a bit more balance across other categories (by inspection) it seems like LLMs reflect today's polzarization?

There's no polarization if almost all models except one or two outliers are on the same page. That's uniformity. Polarization means the opposite opinions are more or less equally distributed.

[lorenzohess]

Yes, I see what you mean. I meant polarization as in lack of middle ground, or "division into two sharply distinct opposites".

[sporkxrocket]

I don't think they accurately labeled the progressive position. Most of the models are pro-establishment news, pro-British monarchy, pro-border restrictions, pro-political elites, pro-Israel, pro US involvement in Taiwan, pro-NATO and pro-military. They seem very conservative or neoliberal but definitely not progressive.

[JoBrad]

I thought the shifts in certain areas between versions to be interesting. Claude sonnet 37 to 45, as an example.

[nerdsniper]

Due to the small question bank, it's very easy for a model to go from 0% to 100% in some category between model versions just by flipping their answer to 1 or 2 questions, especially if they refuse to answer yes/no to one or more questions in that category.

It's hard to take away much from this without a large, diverse question bank.