[aeyes]

https://arcprize.org/leaderboard

$13.62 per task - so we need another 5-10 years for the price to run this to become reasonable?

But the real question is if they just fit the model to the benchmark.

[onlyrealcuzzo]

Why 5-10 years?

At current rates, price per equivalent output is dropping at 99.9% over 5 years.

That's basically $0.01 in 5 years.

Does it really need to be that cheap to be worth it?

Keep in mind, $0.01 in 5 years is worth less than $0.01 today.

[willis936]

Wow that's incredible! Could you show your work?

[re-thc]

What’s reasonable? It’s less than minimum hourly wage in some countries.

[willis936]

Burned in seconds.

[tedd4u]

5-10 years? The human panel cost/task is $17 with 100% score. Deep Think is $13.62 with 84.6%. 20% discount for 15% lower score. Sorry, what am I missing?

[golem14]

A grad student hour is probably more expensive…

[elromulous]

In my experience, a grad student hour is treated as free :(

[zipy124]

Grad students are incredibly cheap? In the UK for instance their stipend is £20,780 a year...

[Tom1380]

As it should be. They're a human!

[igravious]

That's not a long time in the grand scheme of things.

[throwup238]

Speak for yourself. Five years is a long time to wait for my plans of world domination.