top | item 16136948

How We Used Machine Learning to win at HQ Trivia

49 points| bdod6 | 8 years ago |mux.com

23 comments

order
[+] jtokoph|8 years ago|reply
To any future hackers: You don't need OCR. HQ has a simple websocket server that will stream the questions and possible answers in real time. Set up an http proxy on your phone to inspect the requests the app is making. You'll find lots of helpful stuff.
[+] applecrazy|8 years ago|reply
Ooh. Sounds interesting. I've taken an OCR approach before (see my profile for post) since I thought the iOS app had cert pinning, but this method takes the cake and (presumably) will be faster in a game situation.
[+] throwaway2016a|8 years ago|reply
That works until the app starts using certificate pinning.
[+] bdod6|8 years ago|reply
Author here: At mux, we experimented using machine learning to predict HQ Trivia answers. We managed to get 80-90% accuracy across a dataset of around 500 questions.

The trickiest questions were relational questions (e.g. What's heavier, a pineapple or a Siamese cat?). Would appreciate any feedback on our approach (and happy to answer questions!).

[+] conanbatt|8 years ago|reply
Time to bring in the big questions. The tortoise is on its back. And you are not helping it. Why?
[+] selectodude|8 years ago|reply
How is this different from coding, say, a wall hack in an online FPS?
[+] calbear81|8 years ago|reply
I thought this was going to be a retrospective from the HQ Trivia team about how they were mediocre given the scaling challenges and hiccups they are facing and then they solved it through ML!
[+] argonaut|8 years ago|reply
This seems pretty misleading, since honestly 99% of the machine learning that goes on here happens when running the questions/answers through Google Search. There are probably millions of man-years of machine learning / information retrieval that have gone into Google Search.
[+] xkcd-sucks|8 years ago|reply
The concept of machine learning is pretty misleading, because it's founded upon billions of man-years of human learning
[+] petercooper|8 years ago|reply
Then we find HQ eventually pivots to being a machine learning research platform once someone invents a perfectly scoring bot ;-)

Joking aside, I'd say HQ Trivia are getting savvier with the questions. A final question the other day was along the lines of "Which two female artists collectively have the same number of Grammys as Beyoncé?" with the answer being "Adele + Madonna", I believe.

[+] bdod6|8 years ago|reply
Yep. I actually mention that specific question in the article as an unsolvable question for machine learning, at least given our current constraints.

Those are generally rare questions though because difficult questions for bots are also difficult questions for humans. HQ can't have too many of those questions without degrading the player experience.

Because of that, I don't think we will ever get beyond 10/11 questions right per game. That still leads to a decent chance at winning at least one game per week though.

[+] nicolashahn|8 years ago|reply
Is there a dataset of past HQ questions and answers?
[+] bdod6|8 years ago|reply
Yes, we have been archiving each game going back to October. We augment the questions and answers though so that we get more relevant results when run our web scrapes.