[wanderingbit]

> And they didn't even bother to test the most important thing. Were the LLM evaluations even accurate!

This is not true; the professor and the TAs graded every student submission. See this paragraph from the article:

(Just in case you are wondering, I graded all exams myself and I asked the TA to also grade the exams; we mostly agreed with the LLM grades, and I aligned mostly with the softie Gemini. However, when examining the cases when my grades disagreed with the council, I found that the council was more consistent across students and I often thought that the council graded more strictly but more fairly.)