[ryuhhnn]

Some very important context that the researchers don't mention: during the same period that they are claiming test scores improved because of phone bans, Florida changed the way they administer standardised tests. Starting in 2024, they switched from doing one end-of-year assessment and started administering more frequent tests throughout the year in order to better gauge a student's progress and provide a tighter feedback loop. (source: https://www.educationadvanced.com/blog/florida-standardized-...)

It's much more likely that simply changing the way they administer these tests had a more significant impact on test scores than phone bans.

[rahimnathwani]

This criticism would be valid if the researchers had studied just one group of schools, and their methodology was just comparing before and after. But that's not all they did. They had two groups of schools, with low/high cell phone use before the ban. Their hypothesis was that the schools with high cell phone use would see a larger change in test scores (as they would have the largest drop in mobile phone usage).

  We then turn to our causal analysis comparing schools with different degrees of apparent pre-ban student cellphone use, after vs. before Florida’s cellphone ban. We show that the ban increased disciplinary incidents and suspensions significantly in the first year, immediately after the district started referring students for disciplinary action for cellphone use infractions. In particular, our difference-in-differences estimates suggest that the ban increased suspension rates by 12 percent (relative to the comparison group mean) and in-school suspension rates by roughly 20 percent in the first year.

There may be other reasons to criticize the paper, of course.

[ecb_penguin]

This was controlled for in the study.

I swear sometimes people only exist to look for flaws in studies they didn't read.

[red-iron-pine]

the general shit quality and lack of reproducibility means that, for many studies, internet punters can spout that shit and chances are its right.

[jobs_throwaway]

> It's much more likely that simply changing the way they administer these tests had a more significant impact on test scores than phone bans.

Why do you think that's more likely?

[ryuhhnn]

Put yourself in the student's shoes: instead of being required to rote memorise every detail and hold that in your head until the end of the year, you are now only required to be assessed at the time that you are learning the material. Do you think you'd fare better on that type of test, or a test done months after you actually studied the material?

One of the first things they teach you in educational research is that standardised test scores are significantly impacted based on how the tests are administered and what the test is actually assessing.

[tootie]

This is also very closely following the pandemic. I'd imagine that massively pollutes their data. I didn't see a comparison to comparable districts that didn't implement a ban.

Just from anecdata of my own kids, enforcement is nearly impossible. Phones are banned citywide as of this year but it sounds like they are still being used pretty openly.

[etskinner]

Since when are cities banning phones? And how?

[beastman82]

In other words, correlation does not imply causation

[ecb_penguin]

And of course, sometimes a correlation does in fact imply a causation!