They are designed to be evaluated quickly, and objectively. Scantrons can be graded mechanically, and these essays described in the article can be graded with no thought and a minimum of judgment. These goals, efficiency and objectivity, impose constraints on how you can test competence, and as far as I can tell those constraints are simply insurmountable.
I have taken one and exactly one category of standardized test that I respected, and that was the AP tests. The essay questions for AP Lit was graded by having three English teachers read the essay and evaluate whether your answer indicated you understood the meaning of the passage, and how it relates to the work as a whole. In other words, it was done the hard way. And it was better than any other standardized test I've ever taken at correlating performance with understanding.
The problem is scale. Scaling is the only advantage of the current form of tests, and that's enough. Any replacement is going to have to address scaling.
(My hare-brained solution: Grading essays is what teachers do over summer vacation. It's enough excess labor to remove that design constraints.)
Rather than worry about scaling, why not get rid of the essay sections completely? I have an issue with standardized tests basically setting public school curriculum, but most of the essays I remember didn't even do that (and based on the OP's piece, these essays aren't testing writing ability either.)
I recall the (Texas) essay prompts used to be incredibly vague and had nothing to do with what we were learning in class at the time. Prompts like "Write about a time you overcame a challenge" come to mind. Back then, I assumed they were just testing our ability to communicate ideas to an audience, but really all they wanted was the default 5-paragraph essay. It was little more than fill-in-the-blanks to this outline, rather than actual writing.
What made the AP Literature and Rhetoric tests so good was that everything we learned in class was applicable to them. Obviously, my teachers took time to explicitly practice the AP essays, but every other activity also helped feed my knowledge of the subject that I could use to answer the essays on the AP test. Other standardized test essays just seem to be antithetical to any possible purpose.
>Grading essays is what teachers do over summer vacation
One of the few perks of being a teacher is the extra vacation time compared to the rest of us. Cutting into that, with the typically already-poor wages, doesn't seem like a good plan.
And not only for grading, but for teaching as a whole. There is just no smart solution for educating children at a larger scale - we have to do "brute force". The less children per teacher, the better.
(Different story for adults who can autodidact, I guess, but even at universities the student / prof ratio is important).
Points out that one goal is obedience; and even "stupidity" in the system is useful, in that if you're willing enough to go along with obviously stupid orders, you'll pass through to the next level. (In other words, it filters for obedience.)
Obviously, an educational system reflects the distribution of power in a society. This is particularly obvious when we observe an official enemy nation; we have no trouble seeing how they try indoctrinating students along the interests of those with power. Unfortunately, we're taught to have blindspots when it comes to our own societies.
There are of course better educational systems. Unlike the model of dumping knowledge into your empty head, they focus on encouraging the growth of your natural capacities and internal forces. I suspect that many "autodidacts" are just people who want to escape the problems of dominant education, with whatever resources they have.
This is pretty typical. Slamming the standardised tests, without actually giving a better alternative. Maybe instead of complaining about this, these teachers should get together and figure out how to keep from letting so many kids fail out of the educational system.
Good, bad or indifferent this was the plan of the "no child left behind" initiative (which had very strong bi-partisan support mind you) which is now heavily under fire for other reasons.
Its too bad most of the local governments believe throwing money at the situation is the solution. In the Minneapolis School Districts, they're spending close to $13K per student. By comparison, in a suburban school district, they're actually just over $9K per student.
The Minneapolis graduation rate? 49%. The suburban school? 98%.
An alternative to what? Standardized testing? How about, non-standard testing? Do we have such a dearth of qualified teachers that we think we need to strip the people in charge of the classroom of any discretion? What sort of cognitive dissonance does it take to say, "Gee, this person can teach our kids the material, but they probably have no ability to evaluate whether the kids learned it".
Thanks for finding those. I looked through the 2010 test and the essay questions specifically say "Use details from the passage to support your answer." Details, plural, meaning two or more. I'm not seeing essay grading instructions, but I'm leaning towards thinking this is a bit overblown.
Or, in the alternative, learn the rubric, take the silly test according to that rubric (it looks to me as if any student who is reasonably bright and really well taught could rapidly learn to produce answers that fit that rubric), and then write a really well crafted proposal, based on research, for how to make the test better. An eighth grade class that produced a lot of students capable of doing that would be very impressive and would get a lot of attention.
I think it is simpler than this, we take an action that we can both take and demonstrate having taken it, whether or not that solves the problem.
If you are a politician and your constituents are telling you to 'fix education' you can't create a 'Khan Academy' what you can do is add requirements that teachers 'do better' and since better is subjective you create an artificial measure of 'better.' Then you report back that you helped to 'fix education' when you not only didn't fix it, you didn't even move the ball down the field.
However this mythical representative didn't have take the hard road of pissing off some entrenched interests in education in order to change it. Not only would that cut into next year's re-election budget, there would be nothing concrete to show for it and a lot of soft money ads in your district saying how you were bad for education.
I worked for a publisher of standardized tests for a number of years, though this company did not develop the assessments used by the State of New York. What this teacher doesn't seem to appreciate is the process behind how these tests are written. They are, seemingly by the nature of this society, produced in a manner which can't capture distinctive teaching styles like this woman has.
The process of making a test can start with whatever a state legislature has mandated will be assessed. Mind you, before that, there is all the negotiations and politicking that takes place. Surprising to most, this involves state education leaders, business people, religious folks, politicians, parents, etc. It's a kitchen sink of divergent interests with everyone claiming to have the best interests of the children at the fore.
Once the legislation is in place and a contractor has been secured to aid with development, there are loads more meetings and committees deciding what is appropriate assessment within each subject (math, reading, writing, etc.). Again, there are loads of different people with loads of different interests, all of whom believe they are thinking foremost about the children.
It's also in this stage that whatever research or trends in assessment styles will be considered (though some takes place earlier, too). There's usually a mentality of "You go first" to new assessment techniques. States are more willing to try something if another state has already done something similar and there is publicly available data to support the perceived efficacy.
At the next stage is actual development of the assessment materials. We would split this up, part of it being done in-house and a bunch contracted to teachers around the state. Yes, we tried to get teachers from every district. For the contracted work, this would mean paying teachers to write a number of questions for a specific test (say, fifth-grade math). These people got paid for each question they wrote regardless of the quality or usability of what they submitted.
The worst material to write was probably math simply due to the dry nature of the subject and the fact that creative approaches to math are usually verboten in education here. Reading tests were often the most difficult to develop. The work on the tests wasn't bad, but securing copyright permissions and, often, permission to edit was brutal. If there was a magazine piece, the complications were often much worse because usage rights might have to be secured from multiple parties (publisher, author and photographers).
Mind you, this was also in the late '90s, so the Internet wasn't as useful a tool for tracking down rights holders or potential materials, and email was still a secondary means of communication, definitely behind the phone and often behind the fax, too. Securing rights for all the materials we wanted to use took months just because of how hard it was to find people and communicate with them. And states didn't have much of any budget to pay, so securing rights at minimal cost was a big hurdle. Often, the best pieces were never used due to how much a rights holder wanted.
So questions would come in from all over the state, then we would clean them up. That was multi-layered work. It might mean simple grammar and punctuation fixes, but it also meant correcting the format mandated by the state education departments. For example, when I was doing this work, states would not allow us to put a negative in the question. But there were loads of these types of rules, like making certain there was parallel structure among answer choices, not having any choices significant;y shorter or longer, etc.
Once we had done an initial tightening of the new bank of material, all the teachers we'd contracted and state administrators were brought in for a week of refinement and further development of materials. These were simultaneously productive and political sessions. A lot of work would be done, but there was also a lot of on-site jockeying. Teachers would say things like, "This is a great story, but my kids won't be able to relate to it." State administrators would hear this a few times about a piece and then pull the material from any further consideration, not even pilot testing. Quality was often a secondary consideration to how teachers felt their students would do, and it was sometimes tertiary to other teacher goals (what they believed was important, their personal agendas, etc.).
Those last few steps would then repeat themselves. We would tighten up the work that had been developed, the graphics department would develop accompanying graphics where needed and handle page layout, proofreading was a persistent process, and then we would bring the teachers back in for another review of the nearly final materials.
Then we would do another round of tightening-up the material. Proofing, requesting minuscule tweaks from the graphics department, getting state approval for any substantive change (no matter how minor), etc. That was when we could begin building an actual test using these new materials and existing questions from previous tests. We'd also begin to development the accompanying manuals which instructed the schools how to handle the materials and the teachers how to administer the tests. As you can imagine, these had to be perfect. When you have a 100-page document with loads of instructions around specific details, errors are not permissible.
(I haven't done this work in 12 years, but to this day my eyes proofread nearly everything that comes before them. I can be at a simple restaurant, and the menu will list "pan fried chicken." I instinctively note the missing hyphen.)
Of course, all that development work only went to pilot materials. I don't remember exactly, but a student might take a test that was about 70% questions that counted and the rest were new questions being evaluated. Once the tests came back, data analysis was run on everything, enabling us to see what worked and what didn't. Sometimes a question was too hard or too easy, sometimes one group of people simply had issues with a question. My memory is hazy, but I want to say that about one-third of the questions that were piloted became usable. Maybe 10-20% of them got re-piloted because the data showed a way we could possibly fix the question (e.g., one of the answer choices was too attractive, so a re-write of that might be enough of a fix to make the question worth trying again).
On the other side was the scoring for written questions. We had the state-issued rubrics, and those were our guiding force. I (and others) would train the part-time people we hired to do this scoring. The company I worked for hired these people largely off of a standard bank of psychological assessments. The company owner felt these gave all the information we needed to evaluate these potential employees.
Easily, the biggest challenge was getting scorers to accept the rubrics. A student might write a quality piece about something, but it might have been well off topic or not sufficiently on topic based on what the state wanted to assess. During training sessions, I spent a lot of my time diffusing anger from these people and getting them to focus on the rubrics. Gently humor was key in that regard, and I don't recall anyone proving to be a long-term problem in terms of accepting the rubrics.
The other big challenge to this work was the repetition. Reading the answers to the same questions over and over was mentally challenging for people. I don't blame them. Most kids of a specific age aren't too creative when fed a question for a state test. For example, ask them who is a public figure they admire and why, and you're likely to get the bulk of the answers focusing on just a few people (athletes, popular music stars, etc.). For the written assessments, 10% of the student materials were scored twice (by separate people) to ensure accuracy of grades and as a way to identify issues with potential scorers.
I've tried to refrain from too much commentary, but there is no doubt that the materials developed for tests are beaten down throughout the process by bureaucracy and various interests. It's much like the corporate world when the firm has way too many meetings in the course of developing something and there is a leadership vacuum. Oh, sure, there is a person or two who is technically leading things and may have veto power, but there are far too many diverse interests for anything of distinct quality to emerge.
With one of the states for which my employer did work, a woman like the teacher in the link would be invited to participate in the following year's development. The lead state administrator always referred to this as "getting that person's buy-in." And truthfully, it seemed to work because the teachers brought in for this reason felt like they had a voice in the process. None that I saw seemed to appreciate the depth of the whole process, so they all seemed to think they had made a difference in the development of the tests.
More specific to the author of the link, she seems like she's probably a good teacher, better than most. The education system, especially when it comes to statewide assessments, isn't prepared to appreciably handle outliers like her. She probably knows that. She probably knows the real battle to change these kinds of tests is not one she's prepared to tackle. I don't blame her. Nor do I blame her for making a public critique like she did.
“I told you, didn’t I, about hearing Noam Chomsky speak recently? When the great man was asked about the chaos in public education, he responded quickly, decisively, and to the point: “Public education in this country is under attack.” The words, though chilling, comforted me in a weird way. I’d been feeling, the past few years of my 30-plus-year tenure in public education, that there was something or somebody out there, a power of a sort, that doesn’t really want you kids to be educated. I felt a force that wants you ignorant and pliable, and that needs you able to fill in the boxes and follow instructions. Now I’m sure.”
This speaks to a rather odd state of mind on the part of the teacher. “The great man”?
Believing that the state of education and testing is the result of some powerful villain behind the curtain requires more credulity than simply attributing it to the massive complexity and inertia you describe.
Unfortunately, all the excuses in the world don't change the fact that these tests hurt education and hurt children. The production of these things is a shameful act.
I thought the tests we had in the UK were bad, and they are, but the US obviously has a far greater problem. I hope this changes, because you are hurting your children and damaging your future.
The other big challenge to this work was the repetition. Reading the answers to the same questions over and over was mentally challenging for people. I don't blame them. Most kids of a specific age aren't too creative when fed a question for a state test. For example, ask them who is a public figure they admire and why, and you're likely to get the bulk of the answers focusing on just a few people (athletes, popular music stars, etc.). For the written assessments, 10% of the student materials were scored twice (by separate people) to ensure accuracy of grades and as a way to identify issues with potential scorers.
I'm curious - do you happen to know whether the essay portions ever offered any actual statistical utility above and beyond the multiple choice ones? As in, do they actually measure anything that the multiple choice questions can't?
I spent several years teaching SAT and GRE classes, I always noticed that I could predict people's essay scores pretty accurately based on their multiple choice scores. Percentile ranks always tended to be very close between the sections, which always made me wonder whether there was any point in having the essay at all. I always suspected that since the overall level of competence was so low on the SAT (sadly, the GRE was not all that much better...), anyone that was writing at the level where actual quality of writing matters was already scoring top marks on the essay, so they were essentially unmeasured by the scale.
I realize that when it comes to setting up these tests, "include an essay" is likely a political mandate, because people for some reason think that essay sections are "more fair" to "bad testers" or "fluid thinkers" or something like that (I'm pretty sure ETS was more or less forced to include an essay for this reason), so it probably wasn't even an option not to include them. But I'd be curious to know whether anyone ever looked into whether they actually told you anything you didn't already know. ETS does not provide this data or analysis, otherwise I'd check it myself as it relates to the SAT.
I think a problem is that humane teachers are made to be "outliers". She correctly perceives these standardized tests as an attack against students and decent teachers. From bureaucracies like the ones you describe. If these bureaucracies were more efficient, I don't think that'd improve the situation; probably even strengthen their attacks.
Good teaching consists of intelligently applying basic principles to particular situations, which are themselves understood by subjective criteria. It isn't hard to differentiate a good teacher from a bad one, once you take time to look, though it does sometimes take some imagination and observation.
The tests are efforts to identify good teaching on a general and objective basis. That's probably doomed to fail from the start, and what theoretical potential is available is quickly crushed by the political components of the process.
The real question is why we need general and objective measures in the first place. The answers have nothing to do with education, and everything to do with the governance of educational institutions. That governance is the real problem, and nothing's going to matter until it's addressed.
Our standardized tests lead classes to a sort of "malicious compliance" http://en.wikipedia.org/wiki/Malicious_compliance - the tests are So Important to the schools that everything becomes a cram session, in lieu of actual teaching and exploring ideas.
You see a lot of complaints about standardized tests, but the simple fact of the matter is that standardized tests are not going away. People want a way to evaluate student performance in a way that works across many schools, districts, and states. By definition, that's going to be via standardized testing. Complaining about them will not help.
Make better tests. Teach better around the tests. Those options are fine. But implying that the very idea of standardized testing is constricting is a waste of time. They aren't going anywhere.
This kind of stuff is the backbone of the public education system. John Taylor Gatto outlines very well the six lessons every student is taught. I think this fits well with lesson 5:
In lesson five I teach that your self-respect should depend on an observer's measure of your worth. My kids are constantly evaluated and judged. A monthly report, impressive in its precision, is sent into students' homes to spread approval or to mark exactly -- down to a single percentage point -- how dissatisfied with their children parents should be. Although some people might be surprised how little time or reflection goes into making up these records, the cumulative weight of the objective- seeming documents establishes a profile of defect which compels a child to arrive at a certain decisions about himself and his future based on the casual judgment of strangers.
Self-evaluation -- the staple of every major philosophical system that ever appeared on the planet -- is never a factor in these things. The lesson of report cards, grades, and tests is that children should not trust themselves or their parents, but must rely on the evaluation of certified officials. People need to be told what they are worth.
And keep pushing. Keep writing these letters. Keep not accepting things "because that's just the way it is". Keep getting the news out. Even if it takes 10 or 30 years.
It's worth it to provide real educations for the current youth of society. It means a future worth living in.
The author claims that these stupid and clumsy tests are of "a force that wants you ignorant and pliable, and that needs you able to fill in the boxes and follow instructions".
If we're going to believe in some malign conspiracy hindering the public schools, wouldn't we look for it first in those institutions determining what does and doesn't happen in those schools? If we compare those institutions' stated purposes with their interests and actual behaviors, would we find them self-consistent? admirable? What would happen to their influence if that analysis were performed more deeply and frequently?
And if we imagined the tools of such a force, what would they be? Control over the language of debate, insistence on particular assumptions, a particular orthodoxy of procedure and calculation, prohibition of certain questions as unnecessary or beside the point?
Would such a force be honest about its aims and methods? Or would it seek to obscure them, and claim some different, more popular aims?
We might begin looking for this malign 'fungus' by noting these tests were instituted to establish some accountability. Why exactly was that? And why were these obviously lousy tools chosen -- what alternatives were discussed, and why were they rejected?
Yes, I can very well imagine a force, answering that description, wishing to limit the critical faculties produced by our public schools. I can imagine it very well indeed.
Well, don't link to or quote from the material under discussion, or gives examples of any questions or the like - that might leave your readers informed rather than merely exercised.
Putting aside for the moment that favorite punching bag - standardized tests - I question the rather overwrought tone of the piece, beginning with the title: 'A Test You Need to Fail'. This doesn't seem particularly good advice for students, nor does applauding the kinds of test answers she cites. Anyone should know that a response like "I don't think it applies to either one" with no supporting argument to exhibit the slightest knowledge of the subject would, even must, receive zero credit, with a "SAY WHY!!" scribbled in red in the margin. Students are very good at holding facile opinions out of ignorance, and should not be praised for it. Yes, a good teacher can spin a response like that to gold in the classroom, by eliciting the threads of actual knowledge upon which the opinion hangs, but a test can hardly do so. I can understand a teacher lamenting that she didn't know and pass on to her students that test graders would be looking for facts and not opinions, but it's not the test's fault she didn't. It is hardly "criminal" that standardized tests are designed to be objectively gradable. If the questions are poorly designed - which other commenters seem to have assumed, even though no evidence for it is presented here - isn't that more likely to be the result of mediocrity than of malevolence? I was likewise unconvinced by her citation of Noam Chomsky's remark. However much or little one considers Chomsky a "great man", one thing he is not is an expert on elementary/secondary education. He has opinions, like the rest of us. Why not cite the opinions of Jonas Salk or Stephen Sondheim? Chomsky can, however, be counted upon to state his opinions in stark, emotionally charged language, and a bit of this polemical propensity seems to have rubbed off on the (ex-)teacher.
I hate to sound McCarthyan, but if a teacher recommends her students to fail tests while also name-dropping Chomsky and referring to him as "the great man" in the same text, her ulterior goal can be hardly anything other than subversion of capitalistic system. Sometimes things are more black-and-white than we are willing to believe.
Could a voucher system really be worse than this? It would put all the control back in the hands of the parents. Some would misuse it; we know this. But we have the Internet now. Surely a Yelp-equivalent for voucher schools would quickly identify the good and bad schools. Competition would ensue.
Would it be perfect? No. Some parents would prefer religious indoctrination to actual education. Others simply wouldn't care. But perfection is not the correct standard. What we have now is dismal and getting worse.
"There's a reason education sucks, and it's the same reason that it will never, ever, ever be fixed. ... Because the owners of this country don't want that. ... They don't want a population of citizens capable of critical thinking. ... They want obedient workers, people who are just smart enough to run their machines and do the paperwork, and just dumb enough to passively accept ..." -- George Carlin
I'm definitely not for standardized testing, and I think grade school is more about subsidized daycare than effective learning. But the letter is a little bit extreme when suggesting failure of the test. Any sufficiently intelligent child will be able to express their individuality and creativity outside of that particular testing environment. While writing the test, you can choose to recognize what it is, and supply the formulaic answers expected to do well. Then forget about it and continue being creative an excellent in your other endeavours.
There are many scenarios in life where you are expected to follow a procedure. The procedure may not be ideal, and it may even be completely counterproductive. But you jump through the hoops, and apply yourself to the expected outcome of the procedure if you want to succeed.
If I were advising my child I would tell them it's nonsense, the adults messed it up, but try to do well anyway given the rules and expected answers. Playing along can be an important skill. To be used judiciously.
"ou can compose a “Gettysburg Address” for the 21st century on the apportioned lines in your test booklet, but if you’ve provided only one fact from the text you read in preparation, then you will earn only half credit. In your constructed response—no matter how well written, correct, intelligent, noble, beautiful, and meaningful it is—if you’ve not collected any specific facts from the provided readings (even if you happen to know more information about the chosen topic than the readings provide), then you will get a zero."
I of course wrote Gettysburg addresses routinely in 8th grade English. No doubt the reporters at the Washington Post did, too, which is why I've sometimes had to read a story two or three times to find out who did what to whom. Creativity, ain't it great?
Having said that, I think that the mania for measurement does have little to do with actual instruction.
I have no way to judge this teacher's letter. It would be really helpful if the tests were published online, then I could evaluate if this teacher's waxing poetic has merit. Does anyone have a links to sample questions or a past test?
Everyone likes to praise or bitch about the testing, but has anyone actually seen the test?
Merely complaining about the standard tests does not help. Unless, students and teachers make a protest against it, eg all students decide to score zero marks for a national test. But that is very unlikely to happen, some people are very stubborn about the test system because they get the tricks, they can do well in it.
What a better way of changing the education system is to start a new private school applying a better education system and to get good results in a sense that high percent of students become the top leaders in their arena; to show the education ministry what a better education system is like ( the education ministry is surely aware of the complaint from the public, the only reason they have not changed the system yet is that they really don't what kind of system would be better).
> I will also give you the best advice I can, ... “When they give you lined paper, write the other way.”
What the hell does that even mean? If that is your best advice, you are probably full of bad advice. I cannot comprehend the level of confusion required to think that quote is clever.
I interpreted as "write on the paper at a 90 degree angle to the lines" (aka not the way you should be doing it). This seems to be a standard rehashing of "going against the grain", advice to be different, to not follow expectations. That's not what I would necessarily call bad advice.
The quote is a metaphor. It is clearly not about going the other way on lined paper, but rather the basic principle that one should not blindly follow instructions because someone else says they are right. If what you are told to do is wrong, don't do it.
>What the hell does that even mean? If that is your best advice, you are probably full of bad advice. I cannot comprehend the level of confusion required to think that quote is clever.
It means go against the grain? I can't comprehend the level of ignorance that makes you incapable of understanding one of the classic dystopian novels.
[+] [-] lmkg|14 years ago|reply
They are designed to be evaluated quickly, and objectively. Scantrons can be graded mechanically, and these essays described in the article can be graded with no thought and a minimum of judgment. These goals, efficiency and objectivity, impose constraints on how you can test competence, and as far as I can tell those constraints are simply insurmountable.
I have taken one and exactly one category of standardized test that I respected, and that was the AP tests. The essay questions for AP Lit was graded by having three English teachers read the essay and evaluate whether your answer indicated you understood the meaning of the passage, and how it relates to the work as a whole. In other words, it was done the hard way. And it was better than any other standardized test I've ever taken at correlating performance with understanding.
The problem is scale. Scaling is the only advantage of the current form of tests, and that's enough. Any replacement is going to have to address scaling.
(My hare-brained solution: Grading essays is what teachers do over summer vacation. It's enough excess labor to remove that design constraints.)
[+] [-] mattwrench|14 years ago|reply
I recall the (Texas) essay prompts used to be incredibly vague and had nothing to do with what we were learning in class at the time. Prompts like "Write about a time you overcame a challenge" come to mind. Back then, I assumed they were just testing our ability to communicate ideas to an audience, but really all they wanted was the default 5-paragraph essay. It was little more than fill-in-the-blanks to this outline, rather than actual writing.
What made the AP Literature and Rhetoric tests so good was that everything we learned in class was applicable to them. Obviously, my teachers took time to explicitly practice the AP essays, but every other activity also helped feed my knowledge of the subject that I could use to answer the essays on the AP test. Other standardized test essays just seem to be antithetical to any possible purpose.
[+] [-] sliverstorm|14 years ago|reply
One of the few perks of being a teacher is the extra vacation time compared to the rest of us. Cutting into that, with the typically already-poor wages, doesn't seem like a good plan.
[+] [-] CookWithMe|14 years ago|reply
And not only for grading, but for teaching as a whole. There is just no smart solution for educating children at a larger scale - we have to do "brute force". The less children per teacher, the better.
(Different story for adults who can autodidact, I guess, but even at universities the student / prof ratio is important).
[+] [-] read_wharf|14 years ago|reply
And there's the money quote.
[+] [-] adrianbg|14 years ago|reply
[deleted]
[+] [-] calibraxis|14 years ago|reply
Points out that one goal is obedience; and even "stupidity" in the system is useful, in that if you're willing enough to go along with obviously stupid orders, you'll pass through to the next level. (In other words, it filters for obedience.)
Obviously, an educational system reflects the distribution of power in a society. This is particularly obvious when we observe an official enemy nation; we have no trouble seeing how they try indoctrinating students along the interests of those with power. Unfortunately, we're taught to have blindspots when it comes to our own societies.
There are of course better educational systems. Unlike the model of dumping knowledge into your empty head, they focus on encouraging the growth of your natural capacities and internal forces. I suspect that many "autodidacts" are just people who want to escape the problems of dominant education, with whatever resources they have.
[+] [-] darksaga|14 years ago|reply
Good, bad or indifferent this was the plan of the "no child left behind" initiative (which had very strong bi-partisan support mind you) which is now heavily under fire for other reasons.
Its too bad most of the local governments believe throwing money at the situation is the solution. In the Minneapolis School Districts, they're spending close to $13K per student. By comparison, in a suburban school district, they're actually just over $9K per student.
The Minneapolis graduation rate? 49%. The suburban school? 98%.
[+] [-] kingkilr|14 years ago|reply
[+] [-] wtn|14 years ago|reply
http://www.nysedregents.org/Grade8/EnglishLanguageArts/home....
[+] [-] wizzard|14 years ago|reply
[+] [-] tokenadult|14 years ago|reply
[+] [-] narrator|14 years ago|reply
A. Spending more money indiscriminately. Money always makes everything better. Just look at health care!
B: Getting the federal government to take over and centralize everything.
Creative ideas are almost always ignored unless they are a means of executing strategy A or B.
[+] [-] ChuckMcM|14 years ago|reply
If you are a politician and your constituents are telling you to 'fix education' you can't create a 'Khan Academy' what you can do is add requirements that teachers 'do better' and since better is subjective you create an artificial measure of 'better.' Then you report back that you helped to 'fix education' when you not only didn't fix it, you didn't even move the ball down the field.
However this mythical representative didn't have take the hard road of pissing off some entrenched interests in education in order to change it. Not only would that cut into next year's re-election budget, there would be nothing concrete to show for it and a lot of soft money ads in your district saying how you were bad for education.
[+] [-] NaOH|14 years ago|reply
The process of making a test can start with whatever a state legislature has mandated will be assessed. Mind you, before that, there is all the negotiations and politicking that takes place. Surprising to most, this involves state education leaders, business people, religious folks, politicians, parents, etc. It's a kitchen sink of divergent interests with everyone claiming to have the best interests of the children at the fore.
Once the legislation is in place and a contractor has been secured to aid with development, there are loads more meetings and committees deciding what is appropriate assessment within each subject (math, reading, writing, etc.). Again, there are loads of different people with loads of different interests, all of whom believe they are thinking foremost about the children.
It's also in this stage that whatever research or trends in assessment styles will be considered (though some takes place earlier, too). There's usually a mentality of "You go first" to new assessment techniques. States are more willing to try something if another state has already done something similar and there is publicly available data to support the perceived efficacy.
At the next stage is actual development of the assessment materials. We would split this up, part of it being done in-house and a bunch contracted to teachers around the state. Yes, we tried to get teachers from every district. For the contracted work, this would mean paying teachers to write a number of questions for a specific test (say, fifth-grade math). These people got paid for each question they wrote regardless of the quality or usability of what they submitted.
The worst material to write was probably math simply due to the dry nature of the subject and the fact that creative approaches to math are usually verboten in education here. Reading tests were often the most difficult to develop. The work on the tests wasn't bad, but securing copyright permissions and, often, permission to edit was brutal. If there was a magazine piece, the complications were often much worse because usage rights might have to be secured from multiple parties (publisher, author and photographers).
Mind you, this was also in the late '90s, so the Internet wasn't as useful a tool for tracking down rights holders or potential materials, and email was still a secondary means of communication, definitely behind the phone and often behind the fax, too. Securing rights for all the materials we wanted to use took months just because of how hard it was to find people and communicate with them. And states didn't have much of any budget to pay, so securing rights at minimal cost was a big hurdle. Often, the best pieces were never used due to how much a rights holder wanted.
So questions would come in from all over the state, then we would clean them up. That was multi-layered work. It might mean simple grammar and punctuation fixes, but it also meant correcting the format mandated by the state education departments. For example, when I was doing this work, states would not allow us to put a negative in the question. But there were loads of these types of rules, like making certain there was parallel structure among answer choices, not having any choices significant;y shorter or longer, etc.
Once we had done an initial tightening of the new bank of material, all the teachers we'd contracted and state administrators were brought in for a week of refinement and further development of materials. These were simultaneously productive and political sessions. A lot of work would be done, but there was also a lot of on-site jockeying. Teachers would say things like, "This is a great story, but my kids won't be able to relate to it." State administrators would hear this a few times about a piece and then pull the material from any further consideration, not even pilot testing. Quality was often a secondary consideration to how teachers felt their students would do, and it was sometimes tertiary to other teacher goals (what they believed was important, their personal agendas, etc.).
Those last few steps would then repeat themselves. We would tighten up the work that had been developed, the graphics department would develop accompanying graphics where needed and handle page layout, proofreading was a persistent process, and then we would bring the teachers back in for another review of the nearly final materials.
Then we would do another round of tightening-up the material. Proofing, requesting minuscule tweaks from the graphics department, getting state approval for any substantive change (no matter how minor), etc. That was when we could begin building an actual test using these new materials and existing questions from previous tests. We'd also begin to development the accompanying manuals which instructed the schools how to handle the materials and the teachers how to administer the tests. As you can imagine, these had to be perfect. When you have a 100-page document with loads of instructions around specific details, errors are not permissible.
(I haven't done this work in 12 years, but to this day my eyes proofread nearly everything that comes before them. I can be at a simple restaurant, and the menu will list "pan fried chicken." I instinctively note the missing hyphen.)
Of course, all that development work only went to pilot materials. I don't remember exactly, but a student might take a test that was about 70% questions that counted and the rest were new questions being evaluated. Once the tests came back, data analysis was run on everything, enabling us to see what worked and what didn't. Sometimes a question was too hard or too easy, sometimes one group of people simply had issues with a question. My memory is hazy, but I want to say that about one-third of the questions that were piloted became usable. Maybe 10-20% of them got re-piloted because the data showed a way we could possibly fix the question (e.g., one of the answer choices was too attractive, so a re-write of that might be enough of a fix to make the question worth trying again).
On the other side was the scoring for written questions. We had the state-issued rubrics, and those were our guiding force. I (and others) would train the part-time people we hired to do this scoring. The company I worked for hired these people largely off of a standard bank of psychological assessments. The company owner felt these gave all the information we needed to evaluate these potential employees.
Easily, the biggest challenge was getting scorers to accept the rubrics. A student might write a quality piece about something, but it might have been well off topic or not sufficiently on topic based on what the state wanted to assess. During training sessions, I spent a lot of my time diffusing anger from these people and getting them to focus on the rubrics. Gently humor was key in that regard, and I don't recall anyone proving to be a long-term problem in terms of accepting the rubrics.
The other big challenge to this work was the repetition. Reading the answers to the same questions over and over was mentally challenging for people. I don't blame them. Most kids of a specific age aren't too creative when fed a question for a state test. For example, ask them who is a public figure they admire and why, and you're likely to get the bulk of the answers focusing on just a few people (athletes, popular music stars, etc.). For the written assessments, 10% of the student materials were scored twice (by separate people) to ensure accuracy of grades and as a way to identify issues with potential scorers.
I've tried to refrain from too much commentary, but there is no doubt that the materials developed for tests are beaten down throughout the process by bureaucracy and various interests. It's much like the corporate world when the firm has way too many meetings in the course of developing something and there is a leadership vacuum. Oh, sure, there is a person or two who is technically leading things and may have veto power, but there are far too many diverse interests for anything of distinct quality to emerge.
With one of the states for which my employer did work, a woman like the teacher in the link would be invited to participate in the following year's development. The lead state administrator always referred to this as "getting that person's buy-in." And truthfully, it seemed to work because the teachers brought in for this reason felt like they had a voice in the process. None that I saw seemed to appreciate the depth of the whole process, so they all seemed to think they had made a difference in the development of the tests.
More specific to the author of the link, she seems like she's probably a good teacher, better than most. The education system, especially when it comes to statewide assessments, isn't prepared to appreciably handle outliers like her. She probably knows that. She probably knows the real battle to change these kinds of tests is not one she's prepared to tackle. I don't blame her. Nor do I blame her for making a public critique like she did.
[+] [-] charlieok|14 years ago|reply
This speaks to a rather odd state of mind on the part of the teacher. “The great man”?
Believing that the state of education and testing is the result of some powerful villain behind the curtain requires more credulity than simply attributing it to the massive complexity and inertia you describe.
[+] [-] almost|14 years ago|reply
Unfortunately, all the excuses in the world don't change the fact that these tests hurt education and hurt children. The production of these things is a shameful act.
I thought the tests we had in the UK were bad, and they are, but the US obviously has a far greater problem. I hope this changes, because you are hurting your children and damaging your future.
[+] [-] bermanoid|14 years ago|reply
I'm curious - do you happen to know whether the essay portions ever offered any actual statistical utility above and beyond the multiple choice ones? As in, do they actually measure anything that the multiple choice questions can't?
I spent several years teaching SAT and GRE classes, I always noticed that I could predict people's essay scores pretty accurately based on their multiple choice scores. Percentile ranks always tended to be very close between the sections, which always made me wonder whether there was any point in having the essay at all. I always suspected that since the overall level of competence was so low on the SAT (sadly, the GRE was not all that much better...), anyone that was writing at the level where actual quality of writing matters was already scoring top marks on the essay, so they were essentially unmeasured by the scale.
I realize that when it comes to setting up these tests, "include an essay" is likely a political mandate, because people for some reason think that essay sections are "more fair" to "bad testers" or "fluid thinkers" or something like that (I'm pretty sure ETS was more or less forced to include an essay for this reason), so it probably wasn't even an option not to include them. But I'd be curious to know whether anyone ever looked into whether they actually told you anything you didn't already know. ETS does not provide this data or analysis, otherwise I'd check it myself as it relates to the SAT.
[+] [-] jasonwatkinspdx|14 years ago|reply
That a good teacher is an exceptional case that we simply cannot handle with our bureaucracy is a thought that should terrify you to your core.
We get the quality we demand. Stop apologizing for the inadequate status quo.
[+] [-] calibraxis|14 years ago|reply
[+] [-] ericssmith|14 years ago|reply
Math is one the most fascinating, creative, and empowering pursuits of our species.
[+] [-] chernevik|14 years ago|reply
The tests are efforts to identify good teaching on a general and objective basis. That's probably doomed to fail from the start, and what theoretical potential is available is quickly crushed by the political components of the process.
The real question is why we need general and objective measures in the first place. The answers have nothing to do with education, and everything to do with the governance of educational institutions. That governance is the real problem, and nothing's going to matter until it's addressed.
[+] [-] Vargas|14 years ago|reply
I will quote this.
[+] [-] jes5199|14 years ago|reply
It doesn't have to be that way. There are counterexamples: http://www.theatlantic.com/national/archive/2011/12/what-ame...
[Edit: (Technically it's just Goodhart's Law when there's no malicious intent. Hard to tell the difference sometimes, though. http://lesswrong.com/lw/1ws/the_importance_of_goodharts_law/ ) ]
[+] [-] waterlesscloud|14 years ago|reply
Make better tests. Teach better around the tests. Those options are fine. But implying that the very idea of standardized testing is constricting is a waste of time. They aren't going anywhere.
[+] [-] fleitz|14 years ago|reply
In lesson five I teach that your self-respect should depend on an observer's measure of your worth. My kids are constantly evaluated and judged. A monthly report, impressive in its precision, is sent into students' homes to spread approval or to mark exactly -- down to a single percentage point -- how dissatisfied with their children parents should be. Although some people might be surprised how little time or reflection goes into making up these records, the cumulative weight of the objective- seeming documents establishes a profile of defect which compels a child to arrive at a certain decisions about himself and his future based on the casual judgment of strangers.
Self-evaluation -- the staple of every major philosophical system that ever appeared on the planet -- is never a factor in these things. The lesson of report cards, grades, and tests is that children should not trust themselves or their parents, but must rely on the evaluation of certified officials. People need to be told what they are worth.
http://www.cantrip.org/gatto.html
[+] [-] rubidium|14 years ago|reply
It's worth it to provide real educations for the current youth of society. It means a future worth living in.
[+] [-] chernevik|14 years ago|reply
If we're going to believe in some malign conspiracy hindering the public schools, wouldn't we look for it first in those institutions determining what does and doesn't happen in those schools? If we compare those institutions' stated purposes with their interests and actual behaviors, would we find them self-consistent? admirable? What would happen to their influence if that analysis were performed more deeply and frequently?
And if we imagined the tools of such a force, what would they be? Control over the language of debate, insistence on particular assumptions, a particular orthodoxy of procedure and calculation, prohibition of certain questions as unnecessary or beside the point?
Would such a force be honest about its aims and methods? Or would it seek to obscure them, and claim some different, more popular aims?
We might begin looking for this malign 'fungus' by noting these tests were instituted to establish some accountability. Why exactly was that? And why were these obviously lousy tools chosen -- what alternatives were discussed, and why were they rejected?
Yes, I can very well imagine a force, answering that description, wishing to limit the critical faculties produced by our public schools. I can imagine it very well indeed.
[+] [-] anigbrowl|14 years ago|reply
[+] [-] warmfuzzykitten|14 years ago|reply
[+] [-] bluekeybox|14 years ago|reply
[+] [-] unknown|14 years ago|reply
[deleted]
[+] [-] unknown|14 years ago|reply
[deleted]
[+] [-] unknown|14 years ago|reply
[deleted]
[+] [-] ScottBurson|14 years ago|reply
Would it be perfect? No. Some parents would prefer religious indoctrination to actual education. Others simply wouldn't care. But perfection is not the correct standard. What we have now is dismal and getting worse.
[+] [-] thisisnotmyname|14 years ago|reply
[+] [-] ryanoneill|14 years ago|reply
http://www.youtube.com/watch?v=GseyaEibb_4
[+] [-] grannyg00se|14 years ago|reply
There are many scenarios in life where you are expected to follow a procedure. The procedure may not be ideal, and it may even be completely counterproductive. But you jump through the hoops, and apply yourself to the expected outcome of the procedure if you want to succeed.
If I were advising my child I would tell them it's nonsense, the adults messed it up, but try to do well anyway given the rules and expected answers. Playing along can be an important skill. To be used judiciously.
[+] [-] cafard|14 years ago|reply
I of course wrote Gettysburg addresses routinely in 8th grade English. No doubt the reporters at the Washington Post did, too, which is why I've sometimes had to read a story two or three times to find out who did what to whom. Creativity, ain't it great?
Having said that, I think that the mania for measurement does have little to do with actual instruction.
[+] [-] jcampbell1|14 years ago|reply
Everyone likes to praise or bitch about the testing, but has anyone actually seen the test?
[+] [-] xinliang|14 years ago|reply
[+] [-] jcampbell1|14 years ago|reply
What the hell does that even mean? If that is your best advice, you are probably full of bad advice. I cannot comprehend the level of confusion required to think that quote is clever.
[+] [-] johngunderman|14 years ago|reply
[+] [-] acuity12|14 years ago|reply
[+] [-] tseabrooks|14 years ago|reply
[+] [-] LockeWatts|14 years ago|reply
It means go against the grain? I can't comprehend the level of ignorance that makes you incapable of understanding one of the classic dystopian novels.
[+] [-] LVB|14 years ago|reply
[+] [-] cinquemb|14 years ago|reply
"le contexte est plus fort que le concept"
[+] [-] joelmichael|14 years ago|reply
[+] [-] mistercow|14 years ago|reply