This paper is about differences between the academic community discussing open science and reproducibility. One main takeaway is perhaps unsurprising: the open science community is apparently more collaborative, with a more connected co-authorship network and more use of "prosocial" language in paper abstracts.
The more interesting finding is that there are more women in high status (first or last) author positions in the open science community (when the number of authors is smaller) and there is an increasing trend in this community of women occupying such positions. This would be predicted by theories that STEM is an individualistic enterprise less likely to attract people with communal values. Women are more likely to have communal values and this is often provided as a cultural explanation for gender gaps in STEM. But the open science community is a part of STEM that sees communal practices (specifically, the publication of data and code along with information about findings) as key to improving science. This is in contrast to the reproducibility community which has legitimate criticisms of established scientific practices but does not emphasize pro-social practices in the same way as the open science community.
In sum I think the paper is useful by
1. Showing that while "open science" and "reproducibility" have some superficial similarities they are distinct communities with interesting differences.
2. Showing ways that the open science community seems more collaborative and communal and thus it seems attractive to women (and likely may be this way because women are helping to drive it).
The paper also has some shortcomings. Names are not gender and gender isn't binary. There's a lot of discussion about diversity and team science which honestly doesn't seem to have much to do with the empirical contributions of the paper.
From time to time, I review scientific papers for Machine Learning journals.
In the modern machine learning theoretical papers of the type: We looked in problem A, here is a pattern B and it is the case because <math.>
Nearly all papers that I get for a review are applied papers: We took dataset A, model B, tweaked knob C and our result is better than paper D, but we do not know why. I am fine with this approach. Many good ML ideas were introduced in this way.
But I give reject if authors did not provide the dataset and code that reproduces results.
It is not an issue for other reviewers and my reject is often overridden by editors and papers get accepted, even if there is not way that anyone ever will be able to reproduce it.
My take-away from the abstract is that this is not about open science and reproducibility per se but about academic disciplines that have emerged to study those things.
Some academic fields have serious problems with reproducibility, such as social psychology. The preference for social interaction over reproducibility can in part be explained by subjectivity of academic field.
[+] [-] groceryheist|5 years ago|reply
The more interesting finding is that there are more women in high status (first or last) author positions in the open science community (when the number of authors is smaller) and there is an increasing trend in this community of women occupying such positions. This would be predicted by theories that STEM is an individualistic enterprise less likely to attract people with communal values. Women are more likely to have communal values and this is often provided as a cultural explanation for gender gaps in STEM. But the open science community is a part of STEM that sees communal practices (specifically, the publication of data and code along with information about findings) as key to improving science. This is in contrast to the reproducibility community which has legitimate criticisms of established scientific practices but does not emphasize pro-social practices in the same way as the open science community.
In sum I think the paper is useful by
1. Showing that while "open science" and "reproducibility" have some superficial similarities they are distinct communities with interesting differences.
2. Showing ways that the open science community seems more collaborative and communal and thus it seems attractive to women (and likely may be this way because women are helping to drive it).
The paper also has some shortcomings. Names are not gender and gender isn't binary. There's a lot of discussion about diversity and team science which honestly doesn't seem to have much to do with the empirical contributions of the paper.
[+] [-] dependenttypes|5 years ago|reply
Is this true even when excluding fields and papers where names go in alphabetic order?
[+] [-] johndoe42377|5 years ago|reply
[+] [-] ternaus|5 years ago|reply
From time to time, I review scientific papers for Machine Learning journals.
In the modern machine learning theoretical papers of the type: We looked in problem A, here is a pattern B and it is the case because <math.>
Nearly all papers that I get for a review are applied papers: We took dataset A, model B, tweaked knob C and our result is better than paper D, but we do not know why. I am fine with this approach. Many good ML ideas were introduced in this way.
But I give reject if authors did not provide the dataset and code that reproduces results.
It is not an issue for other reviewers and my reject is often overridden by editors and papers get accepted, even if there is not way that anyone ever will be able to reproduce it.
[+] [-] MaxBarraclough|5 years ago|reply
[+] [-] analog31|5 years ago|reply
[+] [-] austincheney|5 years ago|reply