[cool_dude85]

I think the risk is that there is some systematic difference between those who chose to participate and the overall population of public Montessori kids. For instance, maybe those with high incomes disproportionately chose to participate, and Montessori strengthens learning for this group, but if we could measure the whole population the result is more mixed. It can't be a fully RCT if there's some kind of opt-in provision (which is not to say that an opt-in provision is bad, or a study that is not fully RCT is irrelevant).

[mmooss]

This is a speculative criticism about a hypothetical problem. How random was the study?

[notahacker]

https://www.pnas.org/doi/10.1073/pnas.2506130122 You could have answered your own question by reading the abstract, which makes it clear that the OP's conjecture was correct: the lotteries were random or somewhat random, but the groups which consented to the study were notably different, with the treatment group being richer, more educated and whiter. They did of course attempt to control for this, whether the controls were adequate or omitted other underlying differences is another question.