All of psychology was hit by the Replication Crisis. 1 in 3 papers have results that are unable to be replicated. In any field that'd have seriously wide-reaching implications, but in psychology where so much relies on foundational theories like ToM it is extremely significant. This impact was worsened by the fact that so few psychological researchers choose to attempt to replicate results, due in part by the fact that doing so isn't incentivized by the industry.
Embodied cognition basically has not a single paper able to be replicated [0], so it's out the window.
This isn't to say that ToM is a flawed theory - in general, it's conclusion is probably true. Rather, psychology has some terrible, terrible, mainstream practices - things like outlier elimination and p-value rounding basically make it impossible to replicate a study unless the original author is involved.
But _assuming_, without evidence to the contrary, that the theory is still sound is denying the problem exists. Thankfully there is some research [6] attempting to show where the world is at with regards to this particular theory.
The study the article is about uses the fairly typical false-belief task. We've been using versions of it since '83, so it should be solid. Except it might not work at all when a person has autism [1], and as autistic people generally are aware of others and that others can differ in thought, the task may be flawed or our understanding flawed. Or maybe autism really does mean you lack something that fundamental [+]. There are other inconsistencies with it. [7]
The false-photograph task was developed in part because of the apparent limitations of the false-belief task. Unfortunately, Woodward's results haven't yet been replicated, just relied upon.
Before the crisis hit home, the research was leaning towards people with autism being ToM deficient, but replications don't show statically sound results that say it conclusively or not, thanks to p-value rounding, whilst some more recent research suggests we simply aren't measuring it correctly, and those results may be caused by the coping mechanisms employed. [2][5] Which, if true, suggests that the false-belief task may not actually be stimulating the right parts of any individual's mind, but rather just engaging them in something visually intensive.
A lot of our measurements of ToM, such as when using fMRIs, have been called into question, as they might just be falsely noticing the spatial orientation that happens during visual stimulus. [3] But again, there isn't enough replication to say definitively one way or t'other.
There's also some lesser issues. Some of the strongest ToM research suggests that the origin lies in mirror neurons [4], but the animals studied in the article of this thread don't have them, and animals possibly having ToM is extremely inflammatory within the broader psychological community. If you want someone to try and replicate your study, suggest that a particular animal has ToM. There's about a 50/50 success rate, which is not really encouraging.
Hopefully that's a little bit to chew on. As I've said, ToM probably does work. But we don't currently have the statistics to say it does.
[+] I'm not the right person to judge this. I have autism. Which, if the ToM theories are correct, means I lack the perspective to tell whether or not I have a ToM deficit.
Hey, lovely answer, I will take my time to go through all of this.
I currently research mental representation such as emotions in applied computing which is near-at-hand with ToM and although many of the theories seem intuitive, the replication crisis is something that really bothered me (since I come from a STEM background).
I will read through your post and definitely check the sources, thank you very much!
As for your last point, ToM+Autism might be one of the more interesting aspects of ToM research and as far as I can tell, we are far from conclusive theories.
I'm not defending psychology. But you've inserted your own narrative and interpretation into this group of studies which do not necessarily follow. In addition, although they are still important in a lot of ways this is a fast developing field and some of these studies are fairly old.
Your assertion that [2] and [5] are evidence of problems with the conclusion of [1] does not follow. [2] and [5] are findings of atypical neural activity in specific systems/areas of the brain in response to imitating and experiencing emotional stimuli. [1] is a finding that people with ASD score poorly on a test of theory of mind. It does not at all follow that they are evidence that the result of [1] is incorrect. They could just as easily by a reason why people with ASD are lacking in theory of mind rather than be evidence that the result is incorrect.
It's the same thing with [4]. I also don't see evidence to your claim that great apes do not have them. I only see a lack of any kind of studies on MNS in almost all animals. This is the only excerpt I found on them is in [9], and it says nothing of the sort, and it implies that there haven't been any attempts at observation of MNS in other animals including hominids/apes.
There is nothing on whether or not hominids do not have MNS, and on dolphins there is only (at this time) conjecture. On the other hand animals having mirror neurons but not having theory of mind does not mean that mirror neurons are not necessary for forming theory of mind. It just means that it's more complicated than that. [8] asserts that it is a part of it. In fact MNS is now believed to be a separate system from the ToM system (even if they are related and interact) [10].
Although [3] cast some doubt, it is by no means a conclusive dismissal. It essentially says one area of the brain previously studied probably has several different functions instead of just a social-cognitive one. It shows only that the specifically relevant previous work is less conclusive. It is also from 2007, and has nearly 1000 citing papers. One such paper that is fairly highly cited is [10], which compensates for issues presented by [3] by looking at a different area of the brain. Here is another study which examines a different aspect of the brain [12]. It also cites a few studies which it cites as specifically showing reduced activity in regions in the brain which are part of the ToM/mentalizing system.
It is not obvious to me from the evidence you've provided what the problems with ToM are, especially with regards to people with ASD.
shakna|6 years ago
Embodied cognition basically has not a single paper able to be replicated [0], so it's out the window.
This isn't to say that ToM is a flawed theory - in general, it's conclusion is probably true. Rather, psychology has some terrible, terrible, mainstream practices - things like outlier elimination and p-value rounding basically make it impossible to replicate a study unless the original author is involved.
But _assuming_, without evidence to the contrary, that the theory is still sound is denying the problem exists. Thankfully there is some research [6] attempting to show where the world is at with regards to this particular theory.
The study the article is about uses the fairly typical false-belief task. We've been using versions of it since '83, so it should be solid. Except it might not work at all when a person has autism [1], and as autistic people generally are aware of others and that others can differ in thought, the task may be flawed or our understanding flawed. Or maybe autism really does mean you lack something that fundamental [+]. There are other inconsistencies with it. [7]
The false-photograph task was developed in part because of the apparent limitations of the false-belief task. Unfortunately, Woodward's results haven't yet been replicated, just relied upon.
Before the crisis hit home, the research was leaning towards people with autism being ToM deficient, but replications don't show statically sound results that say it conclusively or not, thanks to p-value rounding, whilst some more recent research suggests we simply aren't measuring it correctly, and those results may be caused by the coping mechanisms employed. [2][5] Which, if true, suggests that the false-belief task may not actually be stimulating the right parts of any individual's mind, but rather just engaging them in something visually intensive.
A lot of our measurements of ToM, such as when using fMRIs, have been called into question, as they might just be falsely noticing the spatial orientation that happens during visual stimulus. [3] But again, there isn't enough replication to say definitively one way or t'other.
There's also some lesser issues. Some of the strongest ToM research suggests that the origin lies in mirror neurons [4], but the animals studied in the article of this thread don't have them, and animals possibly having ToM is extremely inflammatory within the broader psychological community. If you want someone to try and replicate your study, suggest that a particular animal has ToM. There's about a 50/50 success rate, which is not really encouraging.
Hopefully that's a little bit to chew on. As I've said, ToM probably does work. But we don't currently have the statistics to say it does.
[0] https://qz.com/1525854/psychologys-replication-crisis-is-deb...
[1] https://www.sciencedirect.com/science/article/pii/0010027785...
[2] https://www.nature.com/articles/nn1611
[3] https://doi.org/10.1177%2F1073858407304654
[4] https://doi.org/10.1016%2FS1364-6613%2898%2901262-5
[5] https://doi.org/10.1016%2Fj.neuroimage.2011.02.067
[6] https://www.sciencedirect.com/science/article/pii/S235234091...
[7] https://journals.sagepub.com/doi/full/10.1177/09567976124478...
[+] I'm not the right person to judge this. I have autism. Which, if the ToM theories are correct, means I lack the perspective to tell whether or not I have a ToM deficit.
nbeleski|6 years ago
I currently research mental representation such as emotions in applied computing which is near-at-hand with ToM and although many of the theories seem intuitive, the replication crisis is something that really bothered me (since I come from a STEM background).
I will read through your post and definitely check the sources, thank you very much!
As for your last point, ToM+Autism might be one of the more interesting aspects of ToM research and as far as I can tell, we are far from conclusive theories.
jcranberry|6 years ago
Your assertion that [2] and [5] are evidence of problems with the conclusion of [1] does not follow. [2] and [5] are findings of atypical neural activity in specific systems/areas of the brain in response to imitating and experiencing emotional stimuli. [1] is a finding that people with ASD score poorly on a test of theory of mind. It does not at all follow that they are evidence that the result of [1] is incorrect. They could just as easily by a reason why people with ASD are lacking in theory of mind rather than be evidence that the result is incorrect.
It's the same thing with [4]. I also don't see evidence to your claim that great apes do not have them. I only see a lack of any kind of studies on MNS in almost all animals. This is the only excerpt I found on them is in [9], and it says nothing of the sort, and it implies that there haven't been any attempts at observation of MNS in other animals including hominids/apes.
There is nothing on whether or not hominids do not have MNS, and on dolphins there is only (at this time) conjecture. On the other hand animals having mirror neurons but not having theory of mind does not mean that mirror neurons are not necessary for forming theory of mind. It just means that it's more complicated than that. [8] asserts that it is a part of it. In fact MNS is now believed to be a separate system from the ToM system (even if they are related and interact) [10].
Although [3] cast some doubt, it is by no means a conclusive dismissal. It essentially says one area of the brain previously studied probably has several different functions instead of just a social-cognitive one. It shows only that the specifically relevant previous work is less conclusive. It is also from 2007, and has nearly 1000 citing papers. One such paper that is fairly highly cited is [10], which compensates for issues presented by [3] by looking at a different area of the brain. Here is another study which examines a different aspect of the brain [12]. It also cites a few studies which it cites as specifically showing reduced activity in regions in the brain which are part of the ToM/mentalizing system.
It is not obvious to me from the evidence you've provided what the problems with ToM are, especially with regards to people with ASD.
[8] https://www.sciencedirect.com/science/article/pii/S014976340...
[9] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5810456/
[10] https://www.ncbi.nlm.nih.gov/pubmed/19524046
[11] https://www.sciencedirect.com/science/article/abs/pii/S10538...
[12] https://jamanetwork.com/journals/jamapsychiatry/article-abst...