"I was simply saying that all writing about social science need not be presented with the formality and precision of the academic world. There is a place for storytelling, in all of its messiness."
Chabris didn't say that Gladwell was informal or merely imprecise. He said he cited studies that have been proven wrong in the service of a story.
In other words, Gladwell did not merely present a scientific conclusion in an informal, lightly supported manner. He presented false conclusions as if they were true, invoking the authority of "science", because it told a better story.
That is an abuse of science and very worthy of scientists being a little "uncalm" about it.
That's fair, but I found this portion of Gladwell's response interesting:
> Chabris does not like this study. He thinks it involved too few subjects and that its findings were not replicated by a subsequent study. Alter and Oppenheimer disagree. They say that the version of desirable difficulty that they explore has been confirmed on numerous other occasions. This is the kind of intramural argument about the nature and value of evidence that social scientists have all the time.
So who am I supposed to trust here? Who has the higher epistemic ground, Chabris or Alter & Oppenheimer? The problem is that I don't know what the scientific consensus is on the validity of that study. I'm not sure whose side I should be taking.
Are Alter & Oppenheimer bad scientists trying to pass off a poor study as robust scholarship? Or is there a valid difference of opinion here?
> he cited studies that have been proven wrong in the service of a story.
That's a substantial overstatement. Gladwell didn't cite studies that have been proven wrong, he cited studies that have not been consistently reproducible. That's still a problem, but it's a very significantly different one.
I have to say, I always take the criticisms of Gladwell very seriously. I read his first three books several years ago and have constantly felt like a sucker over the years as his detractors have grown louder.
And yet I find his responses very convincing, especially this one and the response to the Ask A Korean! article. I don't know, maybe I'm just easily swayed or maybe it speaks more about how persuasive Gladwell is.
I've read his books as well and definitely feel like I've been lied to. I almost feel like he was playing on stereotypes and I fell for it. Of course things aren't as simple as he presents them in outliers, but man did I fall for it at first.
You "felt like a sucker" and from another comment "feel like I've been lied to"?
Do other people's comments and criticisms make you change your mind about something that easily? If you thought it was a good book when you read it, why would anything anyone else says change that? (Short of, say, Gladwell made up the entire thing or has been rampantly plagiarizing without attribution.)
Gladwell's books are very well-written narrative combining summarized research studies, anecdotal stories and histories to support his overall thesis. It is not a scientific study. He "connects the dots" of what he believes to be the central theme of his book. I haven't seen anywhere that he "lied".
There will always be detractors to any non-fiction author, especially one so popular. (See Nassim Taleb's books as another example.) If you can find a non-fiction book without major detractors, it probably isn't very good.
Superficially, he is quite convincing, but when you look at a given statement or claim in isolation, it's common to find anything from an insufficiency of support, to outright howling fallacy.
It's an interesting illustration of his "narratives often begin in one place and end in another" that his Slate piece begins with calling his detractors ill-read and careless ("I didn’t want to debate them. I just wanted them to read my book all the way to the end."), and ends with the implicit argument that criticizing his work is the same thing as calling his adherents stupid ("I was simply saying that all writing about social science need not be presented with the formality and precision of the academic world...My point was that the people who read my books appreciate this.").
"The kinds of people who read books in America seem to have no problem with my writing. But I am clearly a bee in the bonnet of some of the kinds of people who review books in America."
That's a nice example of the sort of false dichotomy that infuriates Gladwell's critics.
I'm dense today, I think. I'm afraid I don't see the false dichotomy. I see "kinds of people who read books in America" and "some of the kinds of people who review books in America". Is there a 3rd/nth group I'm not seeing - perhaps "people who read books and who also review books"?
Aren't some of these arguments against Gladwell along the same lines as people complaining about Mythbusters' lack of rigor? Well, then I defer to zombie Feynman.
I'm not saying Gladwell is right, but Chabris' argument amounts to pulling one referenced study out of context and saying "SEE HOW TERRIBLE THIS IS." He's complaining about sample size... with a single anecdotal example?
Also, social scientists are kind of cute when they get all angry.
I completely agree that bringing a helpful idea to a broader audience is quite valuable. The problem, however, lies in the combination of lack of precision and lack of attention span that creates artifacts of conventional wisdom that rattle around for years and years without question.
Some day, someone's grandkid is going to be spouting something about "10,000 hours" and not even know where it comes from.
I'd say just the opposite. Zombie Feynman's point isn't "lay off the popularizers". It's that the essence of science is testing ideas.
I haven't read a Gladwell book, let alone bothered to run down all the relevant science. But if Gladwell's legion of critics are right, then he's doing the opposite of Mythbusters. They are teaching people to experiment; he's turning ideas into appealing stories without any interest in testing them.
The difference is that Mythbusters is doing their own (informal,just for fun) scientific investigations, while Gladwell is selectively misrepresenting other people's (serious, rigorous) scientific investigations so people take his essays more seriously.
Although I appreciate Gladwell's argument that he combines anecdotal narrative and scientific observation (usually he provides the former, someone else the latter), I don't think that the average reader of the books is clear on the limitations of this approach. I'm afraid many people come away that Gladwell has discovered some brand new, proven thing, and instead of taking the book as a series of illustrations of an idea Gladwell finds compelling and wants to share, they take it as settled science performed meticulously by the man. That's not a criticism that is limited to Gladwell, either - many 'pop science' books do the same, and are mistakenly understood as hard science. I try to point out problematic articles and books whenever I can, but it's not easy to convince someone that really wants to believe, for instance, that adversity (or borderline insanity) is the driver of creativity. And don't get me started on the misappropriation of neuroscience research.
> I don't think that the average reader of the books is clear on the limitations of this approach
The average reader may not, but at least they've been compelled to pick up a book, read it, synthesize it and draw their own conclusions - either agreeing or otherwise. That's better than nothing.
An above-average reader (whatever that is) might read it and be annoyed. A below-average reader might read it and either not understand it and take it as gospel.
Responding to reasoned concerns about rigor with ad hominems in the Internet age is not a recipe for success.
> My point was that the people who read my books appreciate this. They are perfectly aware of the strengths and weakness of the narrative form. They know what a story can and can’t do, and they understand that narratives sometimes begin in one place and end in another.
The point is that they don't and they aren't and they can't - many people believe that Gladwell is more than a storyteller, and take his scientific gallavantry as truth. It's irresponsible to ignore that fact, but it's necessary for Gladwell's business model to do so - people read his works primarily not because he is a good storyteller, but because they believe he is telling the truth, as backed up by science.
There is something ugly about Gladwell's response. Gladwell insinuates that Chabris has a rabid vendetta against Gladwell and is trying to make a name for himself as an iconoclast. It might be true, but I would have liked the response to have taken a high road.
1) He emphasizes that Chabris has written <bold>three</bold> critiques of him as of date and will probably write a fourth
2) He calls the request for debate silly, making Chabris look agressive.
3) Notes that "I clearly drive Chabris crazy"
Everybody loses when the conversation turns personal.
Is there anything wrong with reading and enjoying Gladwell for his prowess as a storyteller? And not reading his books as scientifically rigorous works? Whatever you feel about Gladwell, he is an extremely effective storyteller.
One of the points that always seems to be lost in these criticisms and rebuttals and debates is that Gladwell spawns conversion. And that's a very good thing. Nothing should be taken as sacrosanct on either side. It's good to prod and examine these issues and bring more views to the table. Hopefully that leads to more questions... not dogma.
Spawning conversations on the basis of poor or misleading evidence is not a worthwhile endeavor. While nothing should be taken as sacrosanct, one needs to have solid evidence or a reasonable argument before they start prodding and examining issues.
Just saying "Hey, what if?" is not an incredible benefit to society if the claims are not properly testable.
> Kahneman understood my book. Why couldn’t these guys?
Writers should not be blaming their readers for misconceptions and misunderstandings.
> The first striking thing about all three of Chabris’ reviews of David and Goliath is how much attention he pays to a study that I mention at the beginning of my chapter on dyslexia.
Many of the readers of pop-science books do not know how to properly read research papers. They don't have access to the libraries; they don't have the stats knowledge; they don't know how to pick out the decent studies from the poor studies.
It'd be good if pop-sci writers were doing this work for the readers, or at least helping readers to do the work.
I agree - I think criticism of a book like this should both acknowledge that it tells a compelling story and may open readers' eyes to new ideas — while at the same time pointing out that those ideas don't necessarily have good scientific grounding because of reasons A, B, and C that the reader would not otherwise encounter, not being scientifically minded or not having access to relevant literature.
Chabris dismisses the Alter & Oppenheimer study because no "exact" replications have been done that "he has found," but then proceeds to give weight to a similar study, which failed to replicate the findings, even though that study itself was not an exact replica, because the test questions were not appropriate for the sample:
"In my review, I criticized Gladwell for describing this experiment at length without also mentioning that a replication attempt with a much larger and more representative sample of subjects did not find an advantage for difficult typefaces. One of the original study's authors wrote to me to argue that his effect is robust when the test questions are at an appropriate level of difficulty for the participants in the experiment, and that his effect has in fact been replicated “conceptually” by other researchers. However, I cannot find any successful direct replications—repetitions of the experiment that use the same methods and get the same results—and direct replication is the evidence that I believe is most relevant."
I'm not sure who to believe here. Legitimate criticism exists, but for every successful author, there is one who seeks to piggy back on their fame by attacking it, and for every sensational scientific study, there is more to be gained by disproving it than by replicating it.
"The kinds of people who read books in America seem to have no problem with my writing. But I am clearly a bee in the bonnet of some of the kinds of people who review books in America."
Which I interpret as: Because I have high sales figures, that clearly proves that experts are easily upset fussbudgets who you shouldn't listen to.
[+] [-] abalone|12 years ago|reply
"I was simply saying that all writing about social science need not be presented with the formality and precision of the academic world. There is a place for storytelling, in all of its messiness."
Chabris didn't say that Gladwell was informal or merely imprecise. He said he cited studies that have been proven wrong in the service of a story.
In other words, Gladwell did not merely present a scientific conclusion in an informal, lightly supported manner. He presented false conclusions as if they were true, invoking the authority of "science", because it told a better story.
That is an abuse of science and very worthy of scientists being a little "uncalm" about it.
[+] [-] md224|12 years ago|reply
> Chabris does not like this study. He thinks it involved too few subjects and that its findings were not replicated by a subsequent study. Alter and Oppenheimer disagree. They say that the version of desirable difficulty that they explore has been confirmed on numerous other occasions. This is the kind of intramural argument about the nature and value of evidence that social scientists have all the time.
So who am I supposed to trust here? Who has the higher epistemic ground, Chabris or Alter & Oppenheimer? The problem is that I don't know what the scientific consensus is on the validity of that study. I'm not sure whose side I should be taking.
Are Alter & Oppenheimer bad scientists trying to pass off a poor study as robust scholarship? Or is there a valid difference of opinion here?
[+] [-] nbouscal|12 years ago|reply
That's a substantial overstatement. Gladwell didn't cite studies that have been proven wrong, he cited studies that have not been consistently reproducible. That's still a problem, but it's a very significantly different one.
[+] [-] jere|12 years ago|reply
And yet I find his responses very convincing, especially this one and the response to the Ask A Korean! article. I don't know, maybe I'm just easily swayed or maybe it speaks more about how persuasive Gladwell is.
[+] [-] deelowe|12 years ago|reply
[+] [-] maxprogram|12 years ago|reply
Do other people's comments and criticisms make you change your mind about something that easily? If you thought it was a good book when you read it, why would anything anyone else says change that? (Short of, say, Gladwell made up the entire thing or has been rampantly plagiarizing without attribution.)
Gladwell's books are very well-written narrative combining summarized research studies, anecdotal stories and histories to support his overall thesis. It is not a scientific study. He "connects the dots" of what he believes to be the central theme of his book. I haven't seen anywhere that he "lied".
There will always be detractors to any non-fiction author, especially one so popular. (See Nassim Taleb's books as another example.) If you can find a non-fiction book without major detractors, it probably isn't very good.
[+] [-] aaronem|12 years ago|reply
It's an interesting illustration of his "narratives often begin in one place and end in another" that his Slate piece begins with calling his detractors ill-read and careless ("I didn’t want to debate them. I just wanted them to read my book all the way to the end."), and ends with the implicit argument that criticizing his work is the same thing as calling his adherents stupid ("I was simply saying that all writing about social science need not be presented with the formality and precision of the academic world...My point was that the people who read my books appreciate this.").
[+] [-] robotresearcher|12 years ago|reply
That's a nice example of the sort of false dichotomy that infuriates Gladwell's critics.
[+] [-] jljljl|12 years ago|reply
[+] [-] ScottWhigham|12 years ago|reply
Genuine question
[+] [-] Finster|12 years ago|reply
http://xkcd.com/397/
I'm not saying Gladwell is right, but Chabris' argument amounts to pulling one referenced study out of context and saying "SEE HOW TERRIBLE THIS IS." He's complaining about sample size... with a single anecdotal example?
Also, social scientists are kind of cute when they get all angry.
[+] [-] sailfast|12 years ago|reply
Some day, someone's grandkid is going to be spouting something about "10,000 hours" and not even know where it comes from.
[+] [-] wpietri|12 years ago|reply
I haven't read a Gladwell book, let alone bothered to run down all the relevant science. But if Gladwell's legion of critics are right, then he's doing the opposite of Mythbusters. They are teaching people to experiment; he's turning ideas into appealing stories without any interest in testing them.
[+] [-] snowwrestler|12 years ago|reply
[+] [-] devindotcom|12 years ago|reply
[+] [-] helipad|12 years ago|reply
The average reader may not, but at least they've been compelled to pick up a book, read it, synthesize it and draw their own conclusions - either agreeing or otherwise. That's better than nothing.
An above-average reader (whatever that is) might read it and be annoyed. A below-average reader might read it and either not understand it and take it as gospel.
What's the problem? Truly.
[+] [-] debacle|12 years ago|reply
> My point was that the people who read my books appreciate this. They are perfectly aware of the strengths and weakness of the narrative form. They know what a story can and can’t do, and they understand that narratives sometimes begin in one place and end in another.
The point is that they don't and they aren't and they can't - many people believe that Gladwell is more than a storyteller, and take his scientific gallavantry as truth. It's irresponsible to ignore that fact, but it's necessary for Gladwell's business model to do so - people read his works primarily not because he is a good storyteller, but because they believe he is telling the truth, as backed up by science.
[+] [-] grimtrigger|12 years ago|reply
1) He emphasizes that Chabris has written <bold>three</bold> critiques of him as of date and will probably write a fourth
2) He calls the request for debate silly, making Chabris look agressive.
3) Notes that "I clearly drive Chabris crazy"
Everybody loses when the conversation turns personal.
[+] [-] EpicEng|12 years ago|reply
[+] [-] the_watcher|12 years ago|reply
[+] [-] snowwrestler|12 years ago|reply
[+] [-] kirklove|12 years ago|reply
[+] [-] jljljl|12 years ago|reply
Just saying "Hey, what if?" is not an incredible benefit to society if the claims are not properly testable.
[+] [-] snowwrestler|12 years ago|reply
[+] [-] DanBC|12 years ago|reply
Writers should not be blaming their readers for misconceptions and misunderstandings.
> The first striking thing about all three of Chabris’ reviews of David and Goliath is how much attention he pays to a study that I mention at the beginning of my chapter on dyslexia.
Many of the readers of pop-science books do not know how to properly read research papers. They don't have access to the libraries; they don't have the stats knowledge; they don't know how to pick out the decent studies from the poor studies.
It'd be good if pop-sci writers were doing this work for the readers, or at least helping readers to do the work.
[+] [-] devindotcom|12 years ago|reply
[+] [-] codex|12 years ago|reply
"In my review, I criticized Gladwell for describing this experiment at length without also mentioning that a replication attempt with a much larger and more representative sample of subjects did not find an advantage for difficult typefaces. One of the original study's authors wrote to me to argue that his effect is robust when the test questions are at an appropriate level of difficulty for the participants in the experiment, and that his effect has in fact been replicated “conceptually” by other researchers. However, I cannot find any successful direct replications—repetitions of the experiment that use the same methods and get the same results—and direct replication is the evidence that I believe is most relevant."
I'm not sure who to believe here. Legitimate criticism exists, but for every successful author, there is one who seeks to piggy back on their fame by attacking it, and for every sensational scientific study, there is more to be gained by disproving it than by replicating it.
[+] [-] wpietri|12 years ago|reply
"The kinds of people who read books in America seem to have no problem with my writing. But I am clearly a bee in the bonnet of some of the kinds of people who review books in America."
Which I interpret as: Because I have high sales figures, that clearly proves that experts are easily upset fussbudgets who you shouldn't listen to.
[+] [-] unknown|12 years ago|reply
[deleted]