I like to throw sports trivia at AI systems to see how well they do on something that's low-stakes and very well-documented. Here's one question Wiyomi generated for me just now about college football:
> What coaching change occurred at USC during the 2010-2011 offseason?
> A. Lane Kiffin left for the Oakland Raiders, and Steve Sarkisian took over
> B. Pete Caroll left for the Seattle Seahawks, and Lane Kiffin took over
> C. Steve Sarkisian left for Washington, and Lane Kiffin took over
> D. Lane Kiffin was fired, and Steve Sarkisian took over
Wiyomi said I was wrong, and that D is the correct answer:
> Lane Kiffin was fired as USC's head coach after the 2010 season, and Steve Sarkisian, who had been the offensive coordinator, took over as the interim head coach before being officially named head coach.
Lane Kiffin's tenure at USC lasted from 2010 to 2013. If I can't rely on these AI systems for the simplest low-stakes tasks, what good are they for?
I gave this a tough test, specifically generating a quiz about the video game football manager, and it failed quite badly.
It would either give 3 absurd/joke answers and one good correct one. Or give 3/4 correct answers and then insist that only one of them was correct and the others were wrong.
It's probably only accurate to specific topics and still requires alot of human validation and input. To me that makes it not that better than asking an AI for suggestions conversationally. Probably worse since you can make more progress and adjustments conversationally.
We built Wiyomi after struggling to stay consistent with language learning apps. Most tools felt either too gamified or too generic. So we made something simple: type a topic (or paste a book’s table of contents), and get AI-generated quizzes instantly.
You can track progress, set reminders, and actually build a habit.
We’d love your feedback, try it out at wiyomi.com!
Hi all thank you for the honest feedbacks, This MVP will solve your problem poorly but your honest feedback will help us make this MVP much much better and works!
the_snooze|10 months ago
> What coaching change occurred at USC during the 2010-2011 offseason?
> A. Lane Kiffin left for the Oakland Raiders, and Steve Sarkisian took over
> B. Pete Caroll left for the Seattle Seahawks, and Lane Kiffin took over
> C. Steve Sarkisian left for Washington, and Lane Kiffin took over
> D. Lane Kiffin was fired, and Steve Sarkisian took over
I chose B because I knew that to be true: https://en.wikipedia.org/wiki/Lane_Kiffin#USC_Trojans
Wiyomi said I was wrong, and that D is the correct answer:
> Lane Kiffin was fired as USC's head coach after the 2010 season, and Steve Sarkisian, who had been the offensive coordinator, took over as the interim head coach before being officially named head coach.
Lane Kiffin's tenure at USC lasted from 2010 to 2013. If I can't rely on these AI systems for the simplest low-stakes tasks, what good are they for?
mangamadaiyan|10 months ago
(Not trying to be sarcastic, this is genuinely the best reason I can think of)
Edit: made a change for completeness.
12907835202|10 months ago
It would either give 3 absurd/joke answers and one good correct one. Or give 3/4 correct answers and then insist that only one of them was correct and the others were wrong.
It's probably only accurate to specific topics and still requires alot of human validation and input. To me that makes it not that better than asking an AI for suggestions conversationally. Probably worse since you can make more progress and adjustments conversationally.
hakimihsan_|10 months ago
You can track progress, set reminders, and actually build a habit.
We’d love your feedback, try it out at wiyomi.com!
brokensegue|10 months ago
menzoic|10 months ago
Pasting the table of contents for a book that the LLM has no access to will lead to a high rate of hallucinations
pryelluw|10 months ago
rrr_oh_man|10 months ago
hakimihsan|10 months ago
rrr_oh_man|10 months ago
bananapub|10 months ago