top | item 10139674

Reliably measuring IQ through commercial puzzle games

93 points| davidiach | 10 years ago |sciencedirect.com

60 comments

order
[+] jasode|10 years ago|reply
Well, reading the PDF was anticlimactic. Why? Because page 2 of the PDF describes the 12 games that were used. They were games that test memorization, puzzle completion, and pattern matching.

In other words, the "commercial video games" used in their research were "brain games" and not Pac-Man, Mario Bros, and Call of Duty, etc.

If you've ever taken a real IQ test where the proxy shows you cards to match or pictures with "missing" data, the 12 video games they used are very similar in spirit. It's not surprising that there is high correlation between the scores of those particular types of video games and real IQ tests.

However, that's not to say there is zero correlation between blockbuster video games like Call of Duty and IQ. Maybe another study will pursue that.

[+] alexvr|10 years ago|reply
Thanks for the warning. I was really hoping to see Halo or Quake or something, because in my experience practicing these games in a casual-competitive way, I hit a performance ceiling. In high school I played Halo nearly every day, and a bright friend of mine, who didn't even own the game, was a surprisingly good player for his limited experience.

On a similar note, I once read that reaction time and IQ are correlated (not sure how strongly), which is interesting because you might expect motor functions like that to be orthogonal to higher-order cognitive abilities.

[+] ansible|10 years ago|reply
However, that's not to say there is zero correlation between blockbuster video games like Call of Duty and IQ. Maybe another study will pursue that.

There's a lot to learn and practice to achieve mastery in those games. And experience in other similar video games will strongly correlate to how quick and high the skill in one particular (popular) game will be achieved.

I was watching a video a while back with these two guys playing QuakeIII. They both knew the map intimately, and kept track of when power-ups would respawn based on when they were picked up. Then they based their strategy on what resources they had, and what resources their opponent had and where the other was likely to go next.

So yeah, we're talking hundreds, if not thousands of hours devoted to mastery in this game. Against an opponent like that, two Quake newbies would score about the same (dying almost instantly), even if one of the newbies was otherwise very smart, and the other very dumb.

[+] peeters|10 years ago|reply
I was hoping for at least a mainstream game that was mainly about problem solving, like Portal.
[+] dang|10 years ago|reply
Ok, we (belatedly) changed "video games" to "puzzle games" in the title here.
[+] chromano|10 years ago|reply
I had to go through an IQ test and a couple of brain teasers for a job application -- I wasn't hired even though I scored well in the IQ test and answered correctly 4 out of 5 of the brain teasers, they were expecting an outstanding result (though they would pay you about $20/hour).

I came to conclusion that IQ tests and brain teasers are just bullshit (in the context above, that is). You usually expect an exceptional result of it, because really, in day to day life you usually complete much more challenges and give good solutions than, lets say, 90% of the population.

[+] Uptrenda|10 years ago|reply
These academic pay walls really are a cancer to learning and human knowledge.
[+] semi-extrinsic|10 years ago|reply
In proper scientific fields, authors post preprint PDFs on freely available sites like arXiv.
[+] mohnishc|10 years ago|reply
A year and a half back, when I was in the 10th grade, I did some similar research on the effects of video games on the cognitive skills of students. I specifically designed games for this research. I wanted to further this study to test commercially available video games, but due to lack of support and finance, I could not (I'm still in high school).

You can go through the paper here if you want: http://www.emerginginvestigators.org/2015/01/gaming-cognitiv... OR http://arxiv.org/abs/1504.01665

TL;DR; of the paper: I divided the class into two groups, one that played the video games and one that did not. Two tests were taken - one before the children were allowed to play games and another at the completion of one week. Children were made to play games for an hour daily. The results showed that gaming does improve cognitive skills. I was particularly surprised by the substantial improvement that it led to in scores.

PS: Anyone who wants to take this further or maybe has ideas for implementing this, please feel free contact me. I would love to help you out on this.

[+] kragen|10 years ago|reply
Maybe a better summary of this paper would be "IQ tests presented on a computer screen produce scores highly correlated to IQ tests presented on paper," since they didn't use what you would normally call "video games."
[+] ryanmarsh|10 years ago|reply
Was thinking this knowledge could be used as a recruiting tool, then immediately this came to mind...

"Greetings, Starfighter. You have been recruited by the Star League to defend the frontier against Xur and the Ko-Dan armada..."

[+] stared|10 years ago|reply
They are already used for that purpose:

"The largest human cognitive performance dataset reveals insights into the effects of lifestyle factors and aging" Daniel A. Sternberg, Kacey Ballard, Joseph L. Hardy, Benjamin Katz, P. Murali Doraiswamy and Michael Scanlon, Front. Hum. Neurosci., 20 June 2013 http://dx.doi.org/10.3389/fnhum.2013.00292

[+] hyperion2010|10 years ago|reply
One addendum here is that you can use them to measure g, but as soon as you start trying to use them to improve g any value of g you measure using them will be incorrect due to learning effects and there is no reliable evidence that playing such games can improve g. Source: wrote my prelims on this.
[+] gaelow|10 years ago|reply
reliably <> accurately

Wow, some psychology professor who read Ender's Game (Probably watched it since, come on, psychology is science at its laziest and the timeframes match) decided to make a quick buck by publishing an online paywalled paper on the "discovery" "for the first time" that people give up signs of their intelligence when observed performing an activity that requires them to use it (which matches the description of an infinite number of activities...).

Hey, I can reliably measure your intelligence by reading your comments on Hacker News. May I have your cash-money bucks, please? $_$ I have been working hard with hundreds of my students to prove it! For the first time ever! $_$

The fact is that you need specifically designed games for an accurate test. Most video games just won't make the cut. Even a combination of them: take the top 10 from steam. I dare anyone to accurately measure the IQ of their players just by looking at them them play in whatever lab environment they want. An those would be pretty boring. They would be in fact, tests.

[+] stuxnet79|10 years ago|reply
Publish or perish mentality ... or ... good science hindered by a hyperbolic, misleading title. The former is cancerous but the latter can be easily rectified. I myself can't call it because I didn't click the link hoping I would get a TLDR in the comments.
[+] nevinera|10 years ago|reply
We can estimate a metric of intelligence by observing people performing tasks that require thought and problem solving?

O.O

[+] tyrion|10 years ago|reply
> Here we show, for the very first time, that commercial video games can be used to reliably measure individual differences in general intelligence (g). One hundred and eighty eight university undergraduates took part in the study.

Is this a joke? reliably measure ? 188 students? So are you trying to tell me that if I could play Call of Duty with A. Einstein I would be humiliated despite him never having played a video game in his entire life?

[+] psycr|10 years ago|reply
You would do well to learn about effect size and p values. Your common sense guess at what might be reliable is incorrect, which means your doing a disservice to your own understanding of the world.