[tullianus]

The paper is here: https://www.pnas.org/doi/10.1073/pnas.2405334121

No way does this replicate.

* Calling p = 0.062 "marginally significant" (Study 1, child perceivers, adult targets) rings my alarm bells.

* The adult photos used (for studies 1/2/4B) are headshots from an Israeli casting agency. Are there any stage names in use? The child photos are all twins from a twin study the same lab was running. Those seem like, uh, different datasets, with no attempt to control for differences between them. What are the results if you use child actor headshots. What are the results if you use random adult twins. Come on now.

* Any control for real correlations between name and visible features like race or class? The false names in the multiple-choice question were taken at random from the set of names in the dataset. The mean accuracy (for photos of adults) was like 30% - was that more or less even across photos? Or is the average dragged up by a few photos? Imagine an American context: how much of this result is from people correctly identifying the black guy as Darius or Hakeem (or from correctly identifying that the white guy ISN'T Darius or Hakeem)? If you can eliminate one option from a 4-choice question, then your odds are 33%, not 25%.

* The way the results are interpreted (people change their appearance in adulthood in a way that makes them look more like their name) depends entirely on the "we aged up these photos of children using a GAN" experiment, which seems extremely shaky. Also be serious here: do they really think people named Ariel have more lion-y haircuts on average in a way that people subconsciously pick up on? That people named David are gonna look more kingly? Not to shit on their proposed mechanism too much, but get real.

* They preregistered that they were going to look at the differences between men and women, between children and adults, between people who went to religious schools and people who didn't, and between people who knew an example of most of the names used and people who didn't, AND between participants' scores using names they were familiar with vs names they weren't. Lotta comparisons there; I hope they did a Bonferroni correction on the reported p-values.

[tullianus]

God I keep coming back to them using photos of twins for the children. Imagine taking this study. You're shown a photo of a kid. Does he look more like a John, Paul, George, or Ringo*? You click John. Two photos later you're shown the same goddamn kid. Does he look more like a Matthew, Mark, Luke or John*? You figure the test is bugged or something, so you click John again. OOPS, actually those two photos were identical twins, so you're guaranteed to have gotten one wrong! Does this affect the results? Who knows?

     * (imagine Israeli equivalents)

[danielvf]

I agree with you - both the data and the conclusions drawn from that data are fairly out there.

And there's only 16 adult faces used. If even one of these had a skewed name, that would be a bigger effect size than the entire experiment.