WingNews logo WingNews
top | item 45375392

GDPVal: Measuring the performance of our models on real-world tasks

42 points| BGyss | 5 months ago |openai.com

9 comments

order

esafak|5 months ago

They reported the competitors' performance for a change. Especially curious because OpenAI is not first. Kudos?

westurner|5 months ago

"GDPVal: Measuring AI model performance on real world economically viable tasks" (2025) https://cdn.openai.com/pdf/d5eb7428-c4e9-4a33-bd86-86dd4bcf1...

GDP? GlobalGoals ... The Sustainable Development Goals (SDGs) include 17 goals, 169 targets, and over 230 indicators.

For strategic alignment,

Strategic alignment: https://en.wikipedia.org/wiki/Strategic_alignment

Sustainable Development Goals: https://en.wikipedia.org/wiki/Sustainable_Development_Goals

To produce the SDGs, IIUC they clustered the world's problems as an international collaborative exercise; to succeed the MDGs (2000-2015).

Each country voluntarily produces an annual SDG report on their progress on their Targets according to the Indicators.

IMHO, Priorities should include clean energy and AI efficiency, given the growth projections for energy use of AI (and our electrical bills given continued expected supply shortages of energy)

Which real-word SDG tasks can be AI eval'd?

Snuggly73|5 months ago

Apparently producing a react component that returns a piece of html with aria tags set up. Long horizon my ass.

CuriouslyC|5 months ago

Claude's low noise message style and good commonsense baiting people into thinking they can rely on it for hard stuff.