I was a public school teacher that worked in a district that attempted to measure teacher value add. There was limited pay for performance based on that metric. And the district I worked in was majority racial minority and majority living in poverty.
"Using the expiration of preexisting collective bargaining agreements as a source of exogenous variation in the timing of changes in pay, I show that the introduction of flexible pay raised salaries of high-quality teachers, increased teacher
quality (due to the arrival of high-quality teachers from other districts and increased effort), and improved student achievement."
"The main dataset contains information on the universe of Wisconsin teachers,
linked to student test scores to calculate teacher VA."
"Student Test Scores and Demographics.—Student-level data include math and
reading test scores in the Wisconsin Knowledge and Concepts Examination (WKCE
2007–2014) and the Badger test (2015–2016), for all students in grades 3 to 8, as well
as demographic characteristics such as gender, race and ethnicity, socioeconomic
(SES) status, migration status, English-learner status, and disability."
What this study is saying is that paying teachers more for improved performance by their students on standardized tests results in higher standardized test scores of their students. That does not surprise me. In my experience, this system encouraged teaching to the test (focusing the majority of instruction on test prep rather than traditional instruction) and widespread cheating. Both of these efforts raise standardized test scores. The study assumes that standardized test scores are a direct measure of teacher quality and student achievement. In my experience that is not true.
A secondary question is: is the pay increase high enough to induce more talented people to become a teacher?
If all it does is pull higher quality teachers from other places, it is zero sum and all you eventually end up with is the same quality schooling for a higher price.
I do think teachers as a whole should be paid well, but that is a separate discussion.
What if the standardized tests aren't administered by the teacher? That seems like it would be an easy solution to get around the cheating problem.
As for teaching to the test, what's the real downfall of doing that? Colleges use standardized test scores as the biggest or one of the biggest criteria in admission.
The question is whether test scores as a measure of teacher quality is better than the next best alternative. The authors suggest that this would be seniority, but it could also be subjective measures like complaints from parents, likeability, and office politics.
I agree that there are problems with test-based evaluation, but overall I believe it's the least bad solution. It gives teachers a tangible incentive to figure out why what they're doing may not be working and try something different.
The sick idea that profit is the onlyway to motivate humans is historically idiotic. It is an outcome of pathetic efforts to legitimate capitalism, a system of exploitation.
> This paper has studied the effects of the introduction of flexible pay for pub- lic school teachers on the composition of the workforce, teachers’ effort, and
student achievement. A switch away from seniority-based salary schedules toward pay-for-quality in a subset of Wisconsin school districts resulted in high-quality teachers moving to these FP districts and low-quality teachers either moving to dis-tricts which remained with the salary schedules or leaving the public school system altogether. As a result, the composition of the teaching workforce improved in FP districts. Effort exerted by all teachers also increased and, subsequently, test scores improved.
I understand that using test scores makes for easier science, but I grew up with some level of "teach the test" and in my anecdotal experience you turn schools into the dullest cram schools when performance is measured this way. I did fine on tests but I hated it. The only reason I didn't drop out was because I didn't realize I had the agency to do so.
I saw this discussed on Marginal Revolution today as well.
My main question is whether the gains are fundamentally zero-sum, at least in the short term. Some districts implement flexible pay and some don’t, and then the best teachers move to get paid more. And the places that get left behind…?
In the long-run, increased pay ought to lead to more high-quality teachers entering the profession. But in the short term, this scheme seems likely to redistribute talent in ways that reinforce existing patterns of inequality.
An other more direct issue is Goodhart’s law: if teacher compensation is linked to a specific metric, then that metric is what you’ll get. If the metric is test scores, then you’ll see:
- teaching to the test rather than educating
- trying to get rid of left behind, slow, or difficult students (already an issue for generations in test-oriented private institutions — as opposed to the more remedial “last chance” ones)
- ignoring the groups which have the lowest odds of contributing to the metric (which groups it is depends on the weighting / averaging between pupils)
I agree that this sounds like a different kind of voucher/private school resource transfer. Private schools are able to self-select their students and aren't required by law to take ALL students that want to come. This makes it trivial to be both cheaper and show better test results while at the same time removing resources from the students that actually need help.
The problem is that people in power thinking that this is mostly a teacher driven problem. This would be the same approach as saying that a the amount of money a doctor gets paid should be based on how much weight their patient loses (e.g.) Anyone who thinks teaching is so easy should go try it for a month. This kind of approach also misaligns incentives. The fact is we have good teachers and bad teachers, just like every profession. We need to rethink two things for our education system. Is it to just push kids to college or is it to maximize the number of students who can become gainfully employed and self-supporting adults. Then we need to restructure our resources towards that goal, specifically in terms of class size and ability level and extra help. The main change we need to make is to move investment of resources much more towards elementary age where it can have the longest compounding benefit. (Source: I'm from a family of a dozen current and retired public school teachers)
I do agree that if we increase the pay and benefits for ALL teachers, then we'll create incentive to get better quality teachers in the profession and that competition will get better results in the long term.
> But in the short term, this scheme might judt redistribute talent in ways that reinforce existing patterns of inequality.
Aren't you assuming that rich districts with higher grades implement flexible pay? I think it would be the poor districts with worse grades that would be most likely to want to shake things up.
> this scheme seems likely to redistribute talent in ways that reinforce existing patterns of inequality
Only if school districts that implement flexible pay are also those with higher SES indicators. Which could very well be the case, although the study did try to control for these factors.
I do agree that unless you increase teacher pay vis-a-vis other professions, the effect will mostly be to rob Peter to pay Paul.
My sister is a very talented leader/organizer who went to teachers college and would love to teach but the pay isn't good enough for what she could get elsewhere.
Yet she still obsessively talks about how bad the teaching systems is here in Ontario, Canada and follows all the studies.
I could 100% see a large increase in talent/quality jumping into the system and taking over the culture. Or at least heavily influencing it. The gaming for tests is also probably a short term effect as well, the old guard doing what it knows best, just more of it.
But Canada will be the last place this sort of reform would happen. The unions completely dominate discourse.
This does definitely seem like a setting with a large risk for zero-sum outcomes, as the current state of the art seems to only be able to identify the bottom percentiles and then have them discarded. So current understanding hasn't identified why anyone became a good teacher, so we don't know how to make the bad ones better.
This is a problem because even a bad teacher is very likely better than no teacher, so we can't actually discard very many of the bottom teachers before making things worse.
How would it reinforce existing patterns of inequality if low-income school districts move to a flexible pay model, thereby attracting better teachers?
good performance and putting in extra effort should be rewarded over those who just do their job at the minimum effort levels.
That's a fundamental problem with teacher pay, there are no bonusses, there are hardly incentives or room for raises. It's all just tenure based. Once you're in, you're in.
The Texas Education Agency (TEA) awards school districts for things like students earning industry IT certifications. This can be more than $1k per certification.
Teachers know what will happen if they try to excel in these areas: they will be fired. Coworkers explained this to me, but I did not listen.
I explained to a school board member that I was going to try to get my 9th graders certified. He replied that if even one student failed to earn their certification that I would have failed and he would have to fire me. I tried to reason with the idiot, but he made Dilbert's point haired boss seem competent.
I managed to get about half of my students Microsoft certified.
I left that school district in part because my life was hell there. One secretary in particular was very offended because she had to figure out how to spend the school district's money on certification exam vouchers and had to add students to a field trip to compete in an electronics competition. We took 2nd in the state, and I left the school district. Now I'm an adjunct at a community college.
If you want to fix Texas Public Schools, elect competent school board members.
If you want teachers to perform well, reward them instead of punishing them. When a teacher brings state money into the district, give them at least half of that money.
This is an interesting example of how there is a difference betweena policy being a "heat pump" and a "resistive heater", and that both can occur simultaneously.
The problem with "heat pumps" is that they necessitate a "cold" side, from which they "pump the heat" to the "hot" side. Their goal isn't to increase the overall "heat" in the system, it's to move the "hot" all to one side.
A "resistive heater" can add "heat" to the system more evenly, but is less efficient and you won't see "temperatures" rise nearly as quickly.
We frequently put the burden on teachers and “the system” for outcomes, but rarely do we talk about improving our cultural and societal attitudes towards education. If you come from a family and community that does not value education and views it as free child care, how good your teacher is does not matter very much.
If they want to improve teachers performance - reduce the $hit modern society is making them deal with. They're very under paid.
How would you like to deal with corraling 23'ish hyper nut jobs all at different levels? That you have to be their parents, psychologist, DEI, identity fosterer, special education teachers etc.
Funding in many schools isn't there now for specialists to do these roles.... because teachers wanted to earn more after inflation, so cuts made - more responsibilites to teachers.
So the teachers are having to take on the role of many of society's "dump bucket" of stuff that parents should be taking care of.
When I worked in a relatively well to do suburban district I had class sizes up to 35 students. Because I was a math teacher, administration did their best to keep our class sizes under 30, teachers of other core classes were more likely to have 30+ students, and classes that didn't have standardized tests might have more than 35. If I recall correctly, every year I taught 9th graders, who are often hyper nutjobs.
> Funding in many schools isn't there now for specialists to do these roles.... because teachers wanted to earn more after inflation, so cuts made - more responsibilites to teachers.
This seems as though you're implying that it's the greedy teacher's fault that our school systems shove ever-increasing responsibilities upon them. As if the total pot of money in the system is a fixed, immovable sum, and so that if a teacher wants a raise it must mean that something elsewhere has to give.
Is there any shred of evidence to support that kind of thinking, though? We can quibble about salaries and outlier districts all day long if you want, but fundamentally it's not like teachers are the most highly-paid group of workers out there. And governments manage to find money to increase funding to other systems year after year (the police, for example).
While it's difficult to study in a controlled way, any serious, considered attempt to improve the status quo in state schooling is an extremely good idea.
I adverse using metrics for value people, while I do not adverse metrics per se, as a companion information for statistical purposes. The issue with metrics used for something it that they became a target, not a measure anymore. They push toward conformism, witch happen to be antithetic to innovation.
Using metrics to understand at a large scale what happen, if a method work or not, where to improve etc might be useful and harmless. Using them as a way to prize or penalize have regularly very bad results.
The ancient quis custodiet ipsos custodes it's equally valid for metrics, who evaluate those who design the measure?
It seems like the only way to really assess a teachers performance is by evaluating the outcomes of their students over time.
Since teaching career progression in the US is mostly tenure based I've always wondered why there wasn't a more longitudinal approach to their assessment.
The product is the students ability to achieve over time. One batch of good test results doesn't measure anything other than the teachers ability to get good outcomes for the test.
There's a very interesting chapter in the _Freakonomics_ book about catching cheating teachers. The authors use it as an example of incentives gone wrong: to hit their bonuses for high test scores, teachers have cheated in all manners of ways, from helping their students during standardized tests or literally changing their answers after students hand in the papers.
Hot take: you need to define performance and you can't without market feedback. So being about to freely chose a school and direct your money to it has to be the first prerequisite to even thinking about performance based pay. Some bureaucratic performance definition will just have people optimize for that.
The USA has the largest tax haul of any nation on Earth at $5.5 trillion. That the US government squanders that tax haul is indicative of the corruption, incompetence, and malevolence of the US government’s officials and bureaucrats.
More people need to be encouraged to become teachers - full stop.
Its a valid concern that there are some teachers that are not good, and would be weeded out by performance based metrics. However - everyone else at the school knows who the bad teachers are. They're obvious. They're the ones leaving their class unattended. Doing the bare minimum.
These people can't be weeded out right now because there's a shortage of teachers. The need to keep a warm adult body in that room outweighs the need to discipline and then fire teachers that aren't doing their jobs.
As a thought experiment would we also suggest performance based police salaries? Or do we acknowledge that being a cop potentially sucks and that one aspect of making sure we don't only have bad cops is making the job attractive enough that we're not just picking from the bottom of the barrel.
/edit
To be clear - I'm suggesting we encourage more people to become teachers by paying them more and secondarily making them deal with less bullshit.
I hope you intend to encourage more people to become teachers by incentivizing them with higher pay (however that is distributed within the profession). Moral suasion can only get you so far, and is kinda suspect if you're encouraging people to walk into a bad situation.
Hire mature and experienced people as teachers. It is easy to assess someone mature based on their life achievements. No need to attract greedy ones as greed isn't the most desired competence for people who will work with children.
I'd suggest more any familial units with manners and respect - regardless of wealth or race. Lots of white kids who act poorly in rich & poor neighborhoods the teachers don't want to deal with but have to.
Hard to think of an argument why they shouldn’t be.
You can argue the raw materials are something they can’t control but plenty of other industries have the same issue, real estate, hiring, and we still pay them on performance.
Think we’d see accepting poor behavior, teachers writing off or actively bullying students tank if this were implemented properly.
A real estate agent or a hiring manager can still walk away from a really bad property, candidate, etc. A teacher can't walk away from a really bad kid (or really bad parents!) without leaving the profession entirely.
I can not overstate how much I love LLMs. The important bits for those of us with a busy day:
The key findings of the research include:
Higher Salaries for High-Performing Teachers: Districts that adopted the flexible pay scheme offered higher salaries to high-performing teachers, which attracted quality teachers from districts that did not adopt the scheme.
Improved Teacher Quality: The introduction of performance-based pay resulted in improved overall teacher quality in districts that adopted the new pay scheme. This was due to both the attraction of high-quality teachers and increased effort from current teachers.
Enhanced Student Achievement: There was a noticeable improvement in student achievement in districts with flexible pay schemes. The increase in teacher quality and effort directly contributed to better academic outcomes for students.
dwater|1 year ago
"Using the expiration of preexisting collective bargaining agreements as a source of exogenous variation in the timing of changes in pay, I show that the introduction of flexible pay raised salaries of high-quality teachers, increased teacher quality (due to the arrival of high-quality teachers from other districts and increased effort), and improved student achievement."
"The main dataset contains information on the universe of Wisconsin teachers, linked to student test scores to calculate teacher VA."
"Student Test Scores and Demographics.—Student-level data include math and reading test scores in the Wisconsin Knowledge and Concepts Examination (WKCE 2007–2014) and the Badger test (2015–2016), for all students in grades 3 to 8, as well as demographic characteristics such as gender, race and ethnicity, socioeconomic (SES) status, migration status, English-learner status, and disability."
What this study is saying is that paying teachers more for improved performance by their students on standardized tests results in higher standardized test scores of their students. That does not surprise me. In my experience, this system encouraged teaching to the test (focusing the majority of instruction on test prep rather than traditional instruction) and widespread cheating. Both of these efforts raise standardized test scores. The study assumes that standardized test scores are a direct measure of teacher quality and student achievement. In my experience that is not true.
jorvi|1 year ago
If all it does is pull higher quality teachers from other places, it is zero sum and all you eventually end up with is the same quality schooling for a higher price.
I do think teachers as a whole should be paid well, but that is a separate discussion.
brodouevencode|1 year ago
As for teaching to the test, what's the real downfall of doing that? Colleges use standardized test scores as the biggest or one of the biggest criteria in admission.
azinman2|1 year ago
bitshiftfaced|1 year ago
I agree that there are problems with test-based evaluation, but overall I believe it's the least bad solution. It gives teachers a tangible incentive to figure out why what they're doing may not be working and try something different.
gadders|1 year ago
It amazes me that teachers would be so unethical. They seem worse than the general population.
bena|1 year ago
My wife is a special education teacher and so much of her work does not get reflected on a test.
readyman|1 year ago
marviel|1 year ago
> This paper has studied the effects of the introduction of flexible pay for pub- lic school teachers on the composition of the workforce, teachers’ effort, and student achievement. A switch away from seniority-based salary schedules toward pay-for-quality in a subset of Wisconsin school districts resulted in high-quality teachers moving to these FP districts and low-quality teachers either moving to dis-tricts which remained with the salary schedules or leaving the public school system altogether. As a result, the composition of the teaching workforce improved in FP districts. Effort exerted by all teachers also increased and, subsequently, test scores improved.
micromacrofoot|1 year ago
I understand that using test scores makes for easier science, but I grew up with some level of "teach the test" and in my anecdotal experience you turn schools into the dullest cram schools when performance is measured this way. I did fine on tests but I hated it. The only reason I didn't drop out was because I didn't realize I had the agency to do so.
setgree|1 year ago
My main question is whether the gains are fundamentally zero-sum, at least in the short term. Some districts implement flexible pay and some don’t, and then the best teachers move to get paid more. And the places that get left behind…?
In the long-run, increased pay ought to lead to more high-quality teachers entering the profession. But in the short term, this scheme seems likely to redistribute talent in ways that reinforce existing patterns of inequality.
masklinn|1 year ago
- teaching to the test rather than educating
- trying to get rid of left behind, slow, or difficult students (already an issue for generations in test-oriented private institutions — as opposed to the more remedial “last chance” ones)
- ignoring the groups which have the lowest odds of contributing to the metric (which groups it is depends on the weighting / averaging between pupils)
snarf21|1 year ago
The problem is that people in power thinking that this is mostly a teacher driven problem. This would be the same approach as saying that a the amount of money a doctor gets paid should be based on how much weight their patient loses (e.g.) Anyone who thinks teaching is so easy should go try it for a month. This kind of approach also misaligns incentives. The fact is we have good teachers and bad teachers, just like every profession. We need to rethink two things for our education system. Is it to just push kids to college or is it to maximize the number of students who can become gainfully employed and self-supporting adults. Then we need to restructure our resources towards that goal, specifically in terms of class size and ability level and extra help. The main change we need to make is to move investment of resources much more towards elementary age where it can have the longest compounding benefit. (Source: I'm from a family of a dozen current and retired public school teachers)
I do agree that if we increase the pay and benefits for ALL teachers, then we'll create incentive to get better quality teachers in the profession and that competition will get better results in the long term.
slibhb|1 year ago
Aren't you assuming that rich districts with higher grades implement flexible pay? I think it would be the poor districts with worse grades that would be most likely to want to shake things up.
jackcosgrove|1 year ago
Only if school districts that implement flexible pay are also those with higher SES indicators. Which could very well be the case, although the study did try to control for these factors.
I do agree that unless you increase teacher pay vis-a-vis other professions, the effect will mostly be to rob Peter to pay Paul.
dmix|1 year ago
Yet she still obsessively talks about how bad the teaching systems is here in Ontario, Canada and follows all the studies.
I could 100% see a large increase in talent/quality jumping into the system and taking over the culture. Or at least heavily influencing it. The gaming for tests is also probably a short term effect as well, the old guard doing what it knows best, just more of it.
But Canada will be the last place this sort of reform would happen. The unions completely dominate discourse.
SiempreViernes|1 year ago
This is a problem because even a bad teacher is very likely better than no teacher, so we can't actually discard very many of the bottom teachers before making things worse.
kspacewalk2|1 year ago
Think of it as government-performance based teaching quality variations.
>But in the short term, this scheme seems likely to redistribute talent in ways that reinforce existing patterns of inequality.
Only if disadvantaged districts are less likely to switch to performance-based pay. What is the basis for thinking this will happen?
iLoveOncall|1 year ago
throwaway894345|1 year ago
dacryn|1 year ago
That's a fundamental problem with teacher pay, there are no bonusses, there are hardly incentives or room for raises. It's all just tenure based. Once you're in, you're in.
RecycledEle|1 year ago
The Texas Education Agency (TEA) awards school districts for things like students earning industry IT certifications. This can be more than $1k per certification.
Teachers know what will happen if they try to excel in these areas: they will be fired. Coworkers explained this to me, but I did not listen.
I explained to a school board member that I was going to try to get my 9th graders certified. He replied that if even one student failed to earn their certification that I would have failed and he would have to fire me. I tried to reason with the idiot, but he made Dilbert's point haired boss seem competent.
I managed to get about half of my students Microsoft certified.
I left that school district in part because my life was hell there. One secretary in particular was very offended because she had to figure out how to spend the school district's money on certification exam vouchers and had to add students to a field trip to compete in an electronics competition. We took 2nd in the state, and I left the school district. Now I'm an adjunct at a community college.
If you want to fix Texas Public Schools, elect competent school board members.
If you want teachers to perform well, reward them instead of punishing them. When a teacher brings state money into the district, give them at least half of that money.
crzyman|1 year ago
The problem with "heat pumps" is that they necessitate a "cold" side, from which they "pump the heat" to the "hot" side. Their goal isn't to increase the overall "heat" in the system, it's to move the "hot" all to one side.
A "resistive heater" can add "heat" to the system more evenly, but is less efficient and you won't see "temperatures" rise nearly as quickly.
And that both can occur at the same time.
mberning|1 year ago
StefanBatory|1 year ago
czbond|1 year ago
How would you like to deal with corraling 23'ish hyper nut jobs all at different levels? That you have to be their parents, psychologist, DEI, identity fosterer, special education teachers etc.
Funding in many schools isn't there now for specialists to do these roles.... because teachers wanted to earn more after inflation, so cuts made - more responsibilites to teachers.
So the teachers are having to take on the role of many of society's "dump bucket" of stuff that parents should be taking care of.
dwater|1 year ago
drewbug01|1 year ago
This seems as though you're implying that it's the greedy teacher's fault that our school systems shove ever-increasing responsibilities upon them. As if the total pot of money in the system is a fixed, immovable sum, and so that if a teacher wants a raise it must mean that something elsewhere has to give.
Is there any shred of evidence to support that kind of thinking, though? We can quibble about salaries and outlier districts all day long if you want, but fundamentally it's not like teachers are the most highly-paid group of workers out there. And governments manage to find money to increase funding to other systems year after year (the police, for example).
paulryanrogers|1 year ago
robertlagrant|1 year ago
kkfx|1 year ago
Using metrics to understand at a large scale what happen, if a method work or not, where to improve etc might be useful and harmless. Using them as a way to prize or penalize have regularly very bad results.
The ancient quis custodiet ipsos custodes it's equally valid for metrics, who evaluate those who design the measure?
calderwoodra|1 year ago
kspacewalk2|1 year ago
spaceprison|1 year ago
Since teaching career progression in the US is mostly tenure based I've always wondered why there wasn't a more longitudinal approach to their assessment.
The product is the students ability to achieve over time. One batch of good test results doesn't measure anything other than the teachers ability to get good outcomes for the test.
thiago_fm|1 year ago
Even running a study like this inside the same school you would get wrong data and come to the wrong conclusions, just think about it for a second.
Worse is people wasting time in academia researching ideas with so many holes in it.
This is why measuring schools makes no sense, the creators of the study were likely top students at their time, yet they lack some basic common sense.
Grading is broken, first start by coming with a better alternative
rappatic|1 year ago
andy99|1 year ago
pipeline_peak|1 year ago
BirAdam|1 year ago
kspacewalk2|1 year ago
unknown|1 year ago
[deleted]
jmuguy|1 year ago
Its a valid concern that there are some teachers that are not good, and would be weeded out by performance based metrics. However - everyone else at the school knows who the bad teachers are. They're obvious. They're the ones leaving their class unattended. Doing the bare minimum.
These people can't be weeded out right now because there's a shortage of teachers. The need to keep a warm adult body in that room outweighs the need to discipline and then fire teachers that aren't doing their jobs.
As a thought experiment would we also suggest performance based police salaries? Or do we acknowledge that being a cop potentially sucks and that one aspect of making sure we don't only have bad cops is making the job attractive enough that we're not just picking from the bottom of the barrel.
/edit
To be clear - I'm suggesting we encourage more people to become teachers by paying them more and secondarily making them deal with less bullshit.
jackcosgrove|1 year ago
skirge|1 year ago
gadders|1 year ago
I thought one of the causes was unions? qv "rubber rooms" etc.
jader201|1 year ago
alienicecream|1 year ago
czbond|1 year ago
thaumasiotes|1 year ago
unknown|1 year ago
[deleted]
synergy20|1 year ago
whywhywhywhy|1 year ago
You can argue the raw materials are something they can’t control but plenty of other industries have the same issue, real estate, hiring, and we still pay them on performance.
Think we’d see accepting poor behavior, teachers writing off or actively bullying students tank if this were implemented properly.
goda90|1 year ago
mmh0000|1 year ago