top | item 46124179

School cell phone bans and student achievement

212 points| harias | 3 months ago |nber.org

216 comments

order

i_c_b|3 months ago

Back in the late 90s, when I first entered the video game industry to work (when it was quite scruffy, countercultural, and populated by some pretty odd people), one of the first things I encountered was a new co-worker who, next to his giant tower of used Mountain Dew cans, had a black and white TV in his cubicle. This struck me as very odd at that moment in time - as I understood things, obviously the point of work was supposed to be that it was a place where you worked, not a place where you watched TV. (Now, granted, everyone else was playing the recently released Diablo on their work PCs during lunch in network mode, and we were a game studio after all, so my reaction wasn't totally coherent). Still, no one else had a TV, and that guy was young and single with no work-life balance, he was a recent transplant, and it still seemed unusual at the time.

Fast forward 28 years later, and now everyone has an amazing TV in their pocket at all times when they commute, sit in their work space, go out for coffee or lunch, or go sit down in the bathroom, all with a near infinite collection of video via youtube, netflix, and even massive amounts of porn. How little did I know. And that's to say nothing of texting and twitter and reddit and instant messaging and discord and ...

Several years ago, I was working on a college campus, and there were giant corporate-flavored murals beside some of the city blocks students walked, full of happy multicultural clip art people and exciting innovative technological innovation, and adorned with the message, "Imagine a borderless world!" Clearly that message was meant to be rhetorical, not a call to reflection, critique, or reevaluation. There did not seem to be the suggestion that one might imagine the borderless world and then, having done so, decide it was a problem to be corrected.

I wonder a lot, these days, if we're not deep into a Chesterton's Fence situation, where we have to rediscover the hard way the older wisdom about having separate spheres with separate hard constraints and boundaries on behaviors, communities, and communication pathways to facilitate all sorts of important activities that simply don't happen otherwise - something like borders and boundaries as a crucial social technology, specifically about directing attention productively. Phones and tablets are, in their own Turing complete way, portals to a borderless world that pierces the older intentional classroom boundaries.

RegW|2 months ago

In my first job out of university in the 80s, I spent all one night playing Knight Lore on the Spectrum with friends. I failed to get up the next morning. My boss drove across Leeds and to bang on the door to see if I was alright. I needed that job so I stopped playing computer games.

In the 90s a later boss called me out for spending my days attached to the Slashdot firehose. I had sort-of known that it was a wasteful time sink, so I resolved to completely stop using the social media of its time, and have avoided most incarnations of it ever since (but here I am).

As a scouter working with teenagers, I feel that most kids with a supportive backgrounds will tame this beast for themselves eventually, so I hate to make hard "no phones" rules. I would rather they come to terms with this addiction for themselves. I know that some simply won't finish school without strong guidance, but delaying exposure to this might just be worse in the long term.

lo_zamoyski|2 months ago

Part of the lesson is understanding how we got here.

The answer is, of course, liberal hyperindividualism. By that I don't mean "liberal institutions" or respect for the individual person especially in the face of collectivism, but an ideology of antisocial atomization of the self that thrusts the self into subjective godhood. Paradoxically, this makes people more susceptible to control in practice.

Now, ideological and political programs don't fully realize the consequences of their premises instantly. It can take years, decades, centuries for all the nasty errors to manifest and become so conspicuous that they cannot be ignored. The Enlightenment program in our case. And so, in this hyperindividualism, the social order - its layers, its concentric circles, its various rights and demands on the individual that precede the consent of the individual - is all reduced progressively to not only the consensual, but also the transactional. Social bonds and structures evaporate or become fluid and contingent merely on the transactional; commitment and duty are a prison. Consent as the highest and only moral law leads us to relativism, because if all that is needed is consent to make an act moral and good, then naturally what is morally good will vary from person to person, and even minute to minute for a given person. On top of that, consent can be attained through manipulation and power, and so now individuals joust for power to manufacture consent in order to bless their exploitation of others.

The self cannot be limited in any way according to this program, and any residual limits are the lingering chains of some ancient past.

Perhaps most amusing is how so-called "countercultural" movements are anything but. These are typically just advancing the ideological program, not rejecting it. Contradictions between such movements and the status quo often come in the form of a tension between residual cultural features of an earlier age and the greater faithfulness to the trajectory of the program among the countercultural. Typically, conflicts are over power, not belief. And sometimes, the internal contradictions of the program lead to diverging programs that come into conflict.

kragen|3 months ago

The older wisdom was that you worked on the farm with your husband and children for your entire life, breastfeeding while you peeled the potatoes, putting down your spindle to comfort a crying child. Millers lived in the mill; even blacksmiths lived at their smithies. Except for rituals, separate spheres with separate hard constraints was a novelty of the Satanic mills where the Victorian proletariat toiled.

nicbou|2 months ago

I'm trying, but it's so hard!

I put my phone in a drawer. Everything's in silent mode. I have a fully disconnected, distraction-free iPad for reading and writing. Work only happens on the computer. There are no emails on the phone.

Yet, I can't fully disconnect. Every device, every account, every app mixes work stuff and personal life stuff. And software is so sticky! I can't just check one thing without my attention getting stuck on a notification badge, an email, a feed or some other thing that I should not pay attention to right now.

How do you people handle it?

rootusrootus|2 months ago

I had an early experience with a Palm III and a cell modem strapped to it. It was intoxicating. I still find the pull of the phone to be very strong sometimes. It's an ongoing battle to maintain a healthy relationship with it. Such a useful tool, but also a massive time suck if you let it.

greg_V|2 months ago

We were promised a borderless world and instead got one without boundaries.

aa-jv|2 months ago

I've always had a TV or screen of some sort, devoted to background music or light films, just to fill in the void between lines of code. For some, having such light stuff going on is a productivity booster. I once got a dev team that had been struggling to get things finished, well and truly over the finish line, by putting a fat TV in the room, and giving folks the ability to line up their playlists for the day, as long as it wasn't too violent/inappropriate for the workplace.

We side-watched a ton of stuff together as a team - it was great for morale - and we actually shipped stuff, too. Of course the TV eventually became a console for the build server, but it was always available to anyone to put something on in the background, if they wanted to. Definitely a nice way to get a team to be a bit more coherent - as long as whats being played isn't too crazy.

throwawaylaptop|2 months ago

I work in construction saas of a certain kind, and when I visit customers there is a very very clear difference in quality/size/revenue in companies that allow headphones and those that don't.

I'll let you decide which ones you think are doing better.

agumonkey|2 months ago

You could even argue that society is incapable of not running into these cycles of building wisdom and losing it. Our minds are differential.. things that are here have less value, we seek newness no matter what.

georgeecollins|3 months ago

I am also older and I see that my kids don't have certain things that I perceived as disadvantages at the time but may have helped develop useful habits. These things include quiet and boredom, which helped with focus; lack of ready answers or information, which may have helped imagination or generative reasoning.

I think we can recreate these things if and when we need to, but that recreation may be for the elites. I heard an interview with a professor who said he had to reintroduce Socratic exams to get around chat bots and the fact that kids now have very poor handwriting. At an elite school you can do that.

patcon|2 months ago

> rediscover the hard way the older wisdom about having separate spheres with separate hard constraints and boundaries on behaviors

This is something I also believe. Thanks for saying it.

I've been thinking and reflecting a lot on what I've been calling for myself "generative constraint". It's sure as heck not something that is the same for everyone, but I think we all have a set of them that might help us be our best person.

We've universalized constraints and expansivenesses in a way that seems really poor judgement. And yes, there is a capitalist critique in this too, as any good theory should have :)

imgabe|2 months ago

For certain tasks for me, having a movie running while I'm working is more productive. It gives something to take your attention when you have to wait for something without getting sucked in to endless scrolling.

protocolture|2 months ago

"What if coworker I disapproved of but society"

calderwoodra|2 months ago

Article about smartphones being bad? Right to the top.

Generic comment that would fit in the comment section of any of those articles? Right to the top.

I get baited into reading these posts and comments every day - why can't I stop? Probably for the same reason these posts and comments get up votes.

aschla|3 months ago

I'm not particularly old yet, in my mid-thirties, but I reacted like someone much older when I learned kids are allowed to carry around their phones all day at school.

Back in my day (when we walked to school uphill both ways), we weren't allowed to carry around basic flip phones. They had to be in our locker and only used before or after school.

When and why did it become acceptable for much more distracting and stimulating devices to be allowed in class?

japhyr|2 months ago

I was a classroom teacher from 1994-2019, so I watched the transition through the advent of phones until just before Covid. It's not as simple as it seems, for a few reasons.

One, there's the very real pressure from parents to be able to contact their kids when they need to. In the US, regular school shootings have made this a complicated issue to navigate.

Also, it requires much more consistency from school staff than most people realize. If it's top down and not supported by just about everyone, then many teachers and staff find themselves in endless battles. It takes more consistency and clarity of vision, and consistent enforcement than many schools are capable of.

Last, the devices students carry with them are often more capable and reliable than school based technology. So when students need to look something up, it's easy for them to just pull out their device.

Super-addictive devices in a society that's prioritizing many of the wrong things is a hard thing to manage. How many of you would give up your tech salaries to make $40-60k to take on these issues?

jdalgetty|3 months ago

When parents themselves also became addicted and decided it was easier to give their kids phones than to parent them.

AAAAaccountAAAA|3 months ago

I think it is precisely because they are more distracting. When the most addictive thing in phones was the snake game, kids did not bother to insist in using their phones all the time. Now, when you try to tell a pupil to put the phone away, it often results in a huge arguments, so eventually teachers gave up.

phantasmish|2 months ago

We weren't allowed to have any of several different individual devices the functions of which are present in a smartphone. Banning that stuff was more-or-less uncontroversial. Obviously kids in an ordinary classroom shouldn't have instant cameras, and video recorders, and audio recorders, and Walkmen, and radios, and game boys, and TVs, and flashlights, and...

Now we have devices that are all of those things in one and parents will fight you if you try to keep kids from having or using them. Go figure.

What's baffling is why so many more people started thinking all those devices were OK when they're combined into one device. Like, not much of this is novel, we could have had devices that did most of the relevant things a smartphone does, in class. But we didn't because of fucking course they weren't permitted.

_vqpz|2 months ago

It became "acceptable" because the teachers and admin were already on their phones constantly. I went to grade school from 2005-2017, when iPhones came around the adults got them years before kids did, I had numerous teachers that would sit on their phones half the class.

johnnyanmac|2 months ago

Yeah I'm not that far out of high school but my school in the late 00's had a library policy on phones. You can keep them in your pocket, but don't bring them out during school. Otherwise they get taken for the class time, and it escalates from there.

This included recess and pretty much extended to all non-calculator electronic devices, but it was generally more lax when you weren't disrupting someone. I couldn't imagine brazenly having my phone out while a teacher was talking unless it was an emergency.

kelnos|2 months ago

I'm a little older than you (mid forties), and back in my day (when we walked to school uphill, both ways, in the snow, with no shoes), they banned pagers. (And the penalties could be pretty bad, since "only drug dealers have pagers".)

(The thing that annoyed teachers was when we played games on our graphing calculators, which they of course couldn't ban, since the school required them in the first place!)

accrual|2 months ago

In my case phones were just starting to become commonplace, the Razr was the coolest phone to have, we had iPods but not iPhones, etc. Most instructors didn't want to see any phones and would threaten to take them away, so we became skilled at using T9 under the desk or in sweatshirt pockets, etc.

stonemetal12|2 months ago

Never. There has never been a time when it was OK to use a phone in class. What happened is A) Some kids do take their phone out and play with them and either get caught or not B) Something happens and kids record it aka school fight videos. C) giant moral panic that has very little basis in reality.

bluedino|2 months ago

In the 90's only drug dealers had pagers and cell phones, at least in the eyes of the board of education. If you were caught with one you'd be expelled.

ryuhhnn|3 months ago

Some very important context that the researchers don't mention: during the same period that they are claiming test scores improved because of phone bans, Florida changed the way they administer standardised tests. Starting in 2024, they switched from doing one end-of-year assessment and started administering more frequent tests throughout the year in order to better gauge a student's progress and provide a tighter feedback loop. (source: https://www.educationadvanced.com/blog/florida-standardized-...)

It's much more likely that simply changing the way they administer these tests had a more significant impact on test scores than phone bans.

rahimnathwani|2 months ago

This criticism would be valid if the researchers had studied just one group of schools, and their methodology was just comparing before and after. But that's not all they did. They had two groups of schools, with low/high cell phone use before the ban. Their hypothesis was that the schools with high cell phone use would see a larger change in test scores (as they would have the largest drop in mobile phone usage).

  We then turn to our causal analysis comparing schools with different degrees of apparent pre-ban student cellphone use, after vs. before Florida’s cellphone ban. We show that the ban increased disciplinary incidents and suspensions significantly in the first year, immediately after the district started referring students for disciplinary action for cellphone use infractions. In particular, our difference-in-differences estimates suggest that the ban increased suspension rates by 12 percent (relative to the comparison group mean) and in-school suspension rates by roughly 20 percent in the first year.
There may be other reasons to criticize the paper, of course.

ecb_penguin|2 months ago

This was controlled for in the study.

I swear sometimes people only exist to look for flaws in studies they didn't read.

jobs_throwaway|3 months ago

> It's much more likely that simply changing the way they administer these tests had a more significant impact on test scores than phone bans.

Why do you think that's more likely?

tootie|2 months ago

This is also very closely following the pandemic. I'd imagine that massively pollutes their data. I didn't see a comparison to comparable districts that didn't implement a ban.

Just from anecdata of my own kids, enforcement is nearly impossible. Phones are banned citywide as of this year but it sounds like they are still being used pretty openly.

beastman82|2 months ago

In other words, correlation does not imply causation

NatTheBat|2 months ago

To expect children/teens to outsmart big tech companies putting billions of dollars into getting us all addicted to our phone seems... naive. Removing availability to a vice has always been a somewhat effective strategy to mitigate temptation, i.e. food, drugs, etc.

uniqueuid|3 months ago

You have to admit that it's quite clever how they approximate phone use:

> Our identification strategy relies upon our ability to calculate school-specific measures of smartphone activity that we can attribute to students, rather than adults in the building. To do so, we use detailed smartphone activity data from Advan between January 2023 and December 2024 that we link to LUSD schools using point-of-interest coordinates.13 In particular, we focus on the average number of unique smartphone visits (pings) between 9am and 1pm on school days (a common time frame that elementary, middle, and high schools in LUSD are all in session during school days) in the last two months of the 2022-23 school year (right before the ban took effect) and the first two months of the 2023-24 and 2024-25 school years.14 To disentangle student activity from the smartphone activity of teachers/staff, we subtract the average number of unique smartphone visits between 9am and 1pm on teacher workdays (in the same school year) from the same average on regular school days.

HPsquared|3 months ago

I wonder if there's a hidden confounding selection bias here, i.e. "ability of the school to ban phones". This is probably easier in less chaotic schools where the students listen to the teachers, say.

QuercusMax|2 months ago

In our local schools, they don't make *teachers* responsible for enforcing the bans. Students have to keep their phones in a Yondr pouch. If they're caught with their phone it will be confiscated (and require a parent to pick it up), and the administration will give also give additional consequences such as being banned from extracurriculars or school activities like Prom.

My student tells me that in practice many students don't keep their phone in the pouch, but they are very careful about how and when they use them. Many teachers have a "don't ask, don't tell" policy - if I don't see you using the phone, and it's not disruptive, then they don't care.

moduspol|3 months ago

If the teachers and schools cannot implement a phone ban because the students won't listen to them, it might be time to reassess what their purpose is.

apical_dendrite|2 months ago

A lot of schools implement this by having the students put their phones in a locked pouch when they enter the building in the morning. https://www.overyondr.com/phone-locking-pouch

This generally takes it out of the hands of individual classroom teachers.

pavel_lishin|2 months ago

> There were no statistically significant changes in test scores during the first year of the ban, when disciplinary rates were high. During the second year of the ban, in contrast, test scores increased significantly, with positive effects concentrated during the spring semester (scores increased 1.1 percentiles, on average). The researchers suggest that this may be due to the higher stakes of spring tests, which can affect grade advancement and high school graduation. Test score improvements were also concentrated among male students (up 1.4 percentiles, on average) and among middle and high school students (up 1.3 percentiles, on average).

> When comparing high-effect and low-effect schools, the researchers note significant reductions in unexcused absences during the two years following the cell phone ban. They posit that increased attendance could explain as much as half of the test score improvements noted in their primary analysis.

Seems to me like there wasn't a huge improvement, and the improvement seen could easily be attributed to other things, no?

NegativeLatency|3 months ago

Having grown up in the "no cellphones allowed at school" age, and now having a kid, I'm super glad that my local school district is finally banning phones.

There's always going to be exceptions but speaking for myself there's no way I'd be able to resist the allure of a cellphone in class.

harias|3 months ago

Two years after the imposition of a student cell phone ban, student test scores in a large urban school district were significantly higher than before, David N. Figlio and Umut Özek find in The Impact of Cell Phone Bans in Schools on Student Outcomes: Evidence from Florida (NBER Working Paper 34388).

Paper: https://www.nber.org/papers/w34388

graybeardhacker|2 months ago

If I understand the graph correctly, it looks like there's maybe a three percentage point increase. But given there is a 1 percentage point delta in the pre-ban data, that would seem to indicate there is at least a 1 point variation that must be ignored as irrelevant.

I'm not sure how they generated the error bars but that, to me, would suggest the relevant error could be +/- 1 percentage point. Meaning the delta could be at little as two percentage points.

My intuition says cellphone bans would have a positive impact, but I don't think I'd call this data conclusive. I'd want to see more data from earlier and later.

Also, if these are the same students, then test scores might be reflecting increased maturity. If it's different students of the same age, it could be a shift in some extra-educational factors affecting the younger generation.

Too many unknowns and not enough signal.

TimPC|2 months ago

This seems like the single worst way to measure this because by looking at chronological data you get all kinds of temporal effects. A better baseline would be to look at the difference between pre-cell phone ban scores and post cell-hone ban scores compared to other districts where the cell phone ban didn't occur.

I find it completely unremarkable that test scores went up post-COVID and feel it's very hard to tell what is causing what.

cadamsdotcom|2 months ago

2021-2022 vs 2023-2024. Has anyone asked what Covid had to do with it?

RandallBrown|2 months ago

Are students really allowed to be on their phones during class at a lot of schools?

When I was in high school, we didn't have smartphones, but we had game boys, flip phones, and graphing calculators that could play games.

If we were ever caught playing with any of these things we got in trouble. That seemed sufficient at the time, but is that not the case anymore?

nomel|2 months ago

iPads are required in some public school classrooms, in place of textbooks! There no "lockdown mode" that the teacher can enable, to lock to the apps/websites related to the lesson. It's INSANE.

1970-01-01|3 months ago

It's beyond obvious that they should be paying attention to the classroom and not their screen. Future generations will equate our screentime addictions to smoking and drinking. Just putting it down doesn't work. It needs to be out of reach and in certain locations, taken away from us entirely for the betterment of humanity.

doctorpangloss|3 months ago

The increase in scores is really small. It’s 1.1 percentage points.

My interpretation is, the pandemic is a root cause of lower test scores for many reasons, one reason is that kids started using cell phones way more during the pandemic, and that new stuff on the phone (TikTok, let’s be real) causes lower test scores. Reducing usage during school is addressing a real problem, but it’s one of many real problems, and some are way bigger.

eitally|3 months ago

I think you're correct, but another piece of this is that students (especially in HS, but probably also in MS) have realized they can accomplish most assigned tasks in a fraction of the time using online resources (whether copies of old tests, agentic AI, or other sites), so they lean on their phone for this. A big piece of the missing equation here is the fact that home PC/laptop use has also been consistently decreasing, in favor of phones & tablets, causing phones themselves to be even more indispensable for youth.

I read a position paper last week suggesting the solution to this is to take a zero tolerance policy in the classroom and move all course testing back to pencil & paper / bluebooks. I would support that (as a parent of two current high schoolers).

pavon|3 months ago

I think there is a typo in the paper, that was carried over to the article. These two sentences appear to contradict one another as written:

> Interestingly, we observe significantly improved student test scores in the second year of the ban (about 2-3 percentiles higher than the year before the ban) when suspensions revert to pre-ban levels.

> Overall, we show that student test scores improved by 0.6 percentiles, with the ban increasing spring test scores 1.1 percentiles in the second year relative to the spring test right before the ban took effect.

Instead, I think the 1.1 percentile gain should be about the first year, and a 2-3 percentile gain by the second year. That is consistent with the graph.

But yes, a fairly small gain. I agree that much of the gain could be recovering from losses during the pandemic. Also the FAST is a new test that started in the 2022-2023 school year, so some of this could also be due to students and teachers adjusting to the new test and improving over time.

tylermw|3 months ago

When you're dealing with large populations (here, the study include 230,065 students--a very large number), even small shifts due to some treatment can be significant. It is very hard to generate top-down policy interventions that shift the mean of a population in significant ways: if this treatment effect (banning phones) is real, 1.1 points represents a very big policy win that can easily be applied elsewhere. The devil is in the details, however: they exclude some recent data based on the pandemic, but baseline off of 2022-2023, which was still in the throes of the pandemic. The data they show looks to have around a 0.5-1 sigma variation in percentile from 2022-2024, so the shift from the baseline of around 1 to 4 definitely looks significant, but it will be interesting to see if sticks over time.

malshe|2 months ago

Where did you get 1.1 percentage point? The paper uses percentiles.

"Overall, we show that student test scores improved by 0.6 percentiles, with the ban increasing spring test scores 1.1 percentiles in the second year relative to the spring test right before the ban took effect. These positive test score effects are larger for male students (an effect of 1.4 percentiles on the spring test in the second year) and for students in middle and high schools (1.3 percentiles)."

johnnyanmac|2 months ago

There are of course bigger problems, but it seems like phones are one of those "attainable" problems for a school to fix.

Schools aren't exactly much better equipped to make sure parents don't both need to work 50 hours to survive, nor bring housing prices down. They can barely pay their teachers to begin with.

BenFranklin100|2 months ago

Learning requires focus; cell phones destroy focus.

FilosofumRex|2 months ago

The discussion here is so misdirected - It's not the device that's addictive or destructive, it's the exposure of young developing minds to endless, pointless advertising/sponsorship.

Ban advertising to children & youth and the device itself will be harmless

mikemarsh|2 months ago

Since when is a study needed to confirm that enabling a dopamine addiction, especially in developing minds, is a bad idea? Isn't our own direct experience as adults/parents struggling with said addictions enough?

immibis|2 months ago

Since a study is needed to determine if anything is true. Sometimes the study is simple: look out the window and see if it's raining. But this is not one of those.

derelicta|2 months ago

Only slightly tengential, but am so happy to have invested in a remarkable and ony taking (most of) my notes during classes with it (when I dont break the tip and forgetting the replacement ones at home)

est|2 months ago

If phones were totally banned for kids, where would the kid get to know the world?

TVs? Newspapers? Magazines?

Well those are on-way, with limited feedbacks. What's the better alternative except phones?

BenFranklin100|2 months ago

For those of us who grew up before 2000, we somehow managed. I think younger people today will as well.

Kim_Bruning|2 months ago

This is actually very sad. Finally every child had a computer in their pocket, but it turned out to be a locked down attention exploitation machine. I sincerely wish it wasn't.

That's why they're being banned all over. Citizens and governments finally got savvy to how bad these devices can really be, especially in the hands of the inexperienced.

Imagine if history had gone differently. Maybe history still can go differently. A portable computer in the hands of every child. One that actually works for them, not against them.

Are there people already working on this?

(I do know about eg. F-Droid, which is an improvement due to strict curation)

edit: Think of eg 'A Young Lady's Illustrated Primer' : https://en.wikipedia.org/wiki/The_Diamond_Age

sharperguy|2 months ago

I'm thinking of this as I read this thread. How do you solve the problem of social media firehoses while still allowing kids to have access to the vast wealth of information and advice that the internet provides? It hasn't been so many years since many were predicting that the internet will make schools themselves obsolete.

gcanyon|2 months ago

"Differences in test scores"

Anyone know how to interpret that chart?

AndrewDucker|2 months ago

A difference of 1-2%.

Not something particularly worth worrying about.

socalgal2|2 months ago

I can hardly wait for all the WFT->RTO charts :P

danans|2 months ago

> I can hardly wait for all the WFT->RTO charts :P

Work from ... Texas?

lazy_afternoons|2 months ago

I am reasonably confident that Smart Phones will take the route of cigarettes.

Restricted to adults over 21 years of age.

TheOtherHobbes|2 months ago

And then progressively banned in public places.

Which won't work. The model isn't an exact match.

I'd rather see hard content filters in certain contexts than a complete ban.

No social media sites, games, or messaging would remove most of the problem.

But the ultimate driver of addiction is ad-tech, and all of these measures would cause a significant hit to the ad industries.

So in practice it's easier simply to ban phones.

gpt5|3 months ago

This is an area where hacker news shows its weakness. We have:

1. A chart showing a very low increase (1-2 percent)

2. Nothing to control scores rising in every school in America in the last school year (due to reduction of COVID effects).

3. Scores not moving immediately after the ban, but only after the start of a new school year, which means a new cohort of students muddying the data.

Yet the data fits people's biases here (regardless whether it's right or wrong), so the celebrate it and add anecdotes and explanations why it's true.

ryandrake|2 months ago

There's no study that's good enough for HN.

I don't think I've ever seen a science or research article posted here that didn't immediately get picked apart for this or that in the comment section. The methodology is flawed. The data is flawed. The conclusions cannot be drawn. There are confounding variables not accounted for. The sources are questionable. It's become a trope at this point. Either our commenters' standards are way too high, or all of science reporting is deeply flawed.

nineplay|3 months ago

I'd go further and say its a global weakness and unbelievably destructive. The bulk of current discourse today is:

1. Read a headline/tweet/instagram.

2. Decide whether or not it fits in your worldview.

3. Move forward with the confidence that you are better informed than everyone else who agrees/disagrees with it.

You see it everywhere on all sides of all beliefs.

It didn't use to be like this. We used to read articles, we used to read common news sources, we use to not have media overrun with bad actors who know exactly what to say to get the most engagement and solidify people in their own world views.

It's all over HN and I could have hoped there'd be more willingness to say "let me consider the contents and the source before deciding if I accept it". That attitude is just lost and I don't think it will be regained and I think it's the reason we are all in a death spiral.

uniqueuid|2 months ago

Ok here is the crucial part of the paper:

It's a difference in differences design, using individual-level test scores and de-seasonalized data (p. 13). Their wording is:

> Y_igst is the outcome of interest for student i in grade g in school s in time period t, HighAct_s is an indicator for high pre-ban smartphone activity schools, D_t is a series of time period dummies (t = 0 indicates the first period after the ban took effect), δ_s is school fixed effects, and θ_g is grade fixed effects. In this setting, β_t are the parameters of interest, reflecting the difference in the outcome of interest between treatment and comparison schools for each period, with the period before the ban serving as the omitted category, holding grade level constant.

To me some modeling choices seem a bit heavy-handed, but I'm not an economist and could not do better.

uniqueuid|3 months ago

To be fair, all those details are in the paper. And a 1-2 percent increase does not seem low to me for such a measure.

jeffbee|2 months ago

I don't think you are being totally fair to the paper, but you do point out something that drives me crazy: local freakouts about year-to-year changes in math tests scores. It's like people don't realize that these 8th grade students are not the same as those.

JumpCrisscross|2 months ago

The lack of control and cohort following are legitimate criticisms. The effect size is not. Even a single-digit percentage increase over a single year from policy treatment is incredibly impressive when we open the door to cumulative effects.

> Yet the data fits people's biases

It does. But it also fits priors, particularly those we've seen documented when it comes to teens and social media.

fph|2 months ago

There is another major factor that could cause scores to rise globally: ChatGPT. It is now good enough that it can explain topics to students at home, like a private tutor.

tehjoker|2 months ago

Yea, it's strange that the line didn't move quickly. I would give grace for a couple weeks to a few months, but next year? The timing feels really disconnected.

amai|2 months ago

The chart is showing percentiles not percent. Put your smartphone away before posting on HN.

protocolture|2 months ago

Correlation/Causation issues as others have pointed out.

But also, complete inability of schools to adapt.

Refusing to adapt to the reality that is, students will be living their entire lives with these devices, and that they should be working out ways to ensure student productivity despite their existence, is not the same thing as success.

Teachers are inherently lazy. Its one of their more human qualities. But really they need to adapt, or fail and be replaced.

There was a kid in my class in highschool. We had a school that permitted laptops, one of the first near us, but situationally. Teachers could exclude, or instruct the student to not use the laptop for periods during class. However policy was that students were allowed to use the laptop any time they could use a workbook. This kid was the only one who both had access to a laptop and was willing to risk damaging it by bringing it to school.

Math class with this kid was:

1. He plays games on his laptop unless the teacher was looking, in which case he would be solving problems in excel or notepad. Proficient alt tab user.

2. At the end of class, he would copy out all the answers from the back of the textbook to his workbook, and he would hand that in.

English was different. In english the teacher built a relationship with him and engaged him directly. If he was unresponsive he might be forced back to attention somehow, asked a direct question about the text, but that was true of a lot of the students. The entire class was a discussion on book content. When he used the laptop, he was using it to write notes because he was engaged through positive reinforcement. If the teacher caught him playing, the teacher would on those rare occasions, engage him about the game. Often, he had finished his assigned reading\tasks early and simply drifted over. In that case he was left to play because he wasn't disrupting anyone.

Phone bans are a crutch for lazy, uninterested educators. Kids need to be prepared to live in a world with these things in their pockets. The correct dopamine reward feedback loops are not going to be built by banning them entirely. And being better at rote learning and regurgitating ancient course material isn't a strong indicator, if it was even an indicator, of better student outcomes.

kelnos|2 months ago

Perhaps I'm just lacking in imagination, but I'm not sure how having smartphones in class meaningfully enhances the educational experience. I could see a tablet for note-taking being useful, perhaps. (Or a more limited device like a Remarkable, which IIRC isn't a general-purpose touchscreen computer.)

Kids can prepare to live in a world with these things in their pockets outside of school. Sure, primary school is about learning all sorts of things, not just what the teacher is lecturing about. But it doesn't have to be about everything, and I think it's fair to make the call that the distraction caused by smartphones (especially with all the apps designed to addict users) outweighs any positive teachable moments with them present.

Your anecdote is interesting because it didn't really bring me to the same conclusion. Kids aren't going to be interested in every single subject, but we believe it's important to expose them to a bit of everything regardless. Maybe your classmate just didn't care about math, but enjoyed English and reading, and found that much more engaging. Maybe there was absolutely nothing the math teacher could have done to get that kid to pay attention all the time, even if they were the best teacher in the world.

> he would copy out all the answers from the back of the textbook to his workbook, and he would hand that in.

That's really the big flaw here. If he hadn't been able to do that, his grades would suffer, and someone, whether the teacher or his parents, would have (hopefully!) stepped in to see what was going on, and find a way to curb his game-playing in class. Maybe that would have meant he wasn't allowed to have the laptop out in that particular class until he could demonstrate that he could use it appropriately.

> Teachers are inherently lazy.

I know quite a few teachers, and calling teachers lazy is so mind-blowingly, disrespectfully inaccurate that it's really hard to take your opinions seriously. Sure, in every walk of life you'll find lazy people, but I see no evidence that teachers are on average lazier than people in any other profession. My take on it is that teachers might be on average less lazy than your average human.

trgn|2 months ago

Crazy we need studies for this