A new research paper by a team of economists got a lot of pretty favorable press because it appears to deliver results that would seem to confirm what many in the media believe about American schools: If you could just use standardized test scores to weed out underperforming teachers, you would see serious improvement in school achievement.
Media coverage often glosses over the core problem here, which is how you measure teacher performance in the first place. The “value-added” research that is touted by many pundits–using test scores to determine a teacher’s effectiveness–is controversial in large part because critics don’t think it does what its supporters say it does (not to mention that dramatic swings in such scores from year to year, which can make a teacher “great” one year and below average the next). These are rather important criticisms that value-added boosters should engage.
Or they can be New York Times columnist Nicholas Kristof. In Kristof’s first column on the research (1/12/12), he cheered the study’s suggestion that good teachers boost student incomes:
Each of the students will go on as an adult to earn, on average, $25,000 more over a lifetime–or about $700,000 in gains for an average size class–all attributable to that ace teacher back in the fourth grade. That’s right: A great teacher is worth hundreds of thousands of dollars to each year’s students, just in the extra income they will earn.
There have been several interesting critiques written of this, which was written by Harvard’s Raj Chetty and John Friedman and Columbia’s Jonah Rockoff, but has not yet been peer-reviewed or published. At the United Federation of Teachers blog (1/8/12), Leo Casey argues that value-added research
assumes that standardized exams are accurate, reliable and robust measures of actual student learning, a necessary assumption if one is to use them as a measure of teacher performance. It is tautological to claim that an analysis proves what it assumes, especially when that assumption is precisely what is contested in the public debate over standardized tests and value added measures.
Casey goes on to note that the singling out of future earnings–which featured so prominently in the coverage of the study–is also problematic. He cites another critic, education writer and scholar Sherman Dorn, who wrote:
If you want to generalize this claim beyond the data used for the study–associating the group effect scores with teacher quality more generally, making claims about lifetime income, or extrapolating to policy questions–you are making assumptions beyond what the data support.
These are some of the many criticisms of the study. But Kristof’s follow-up column (1/22/12) skipped any serious discussion in favor of this caricature:
After I wrote about the study, skeptics of school reform wrote me to say: Sure, a great teacher can make a difference in the right setting, but not with troubled, surly kids in a high-poverty environment.
Who is arguing that poor, “surly” kids can’t be reached by good teachers?
Kristof then goes on to find a living, breathing rebuttal to an argument no one is making. “Olly Neal was a poor black kid with an attitude,” Kristof tells readers. His life turned around when his teacher, Mildred Grady, started buying books she thought he might enjoy and placing them in the library. That changed Neal’s life–he “caught the book bug,” went to college and eventually became a judge. And thus, Kristof argues:
To me, the lesson is that while there are no silver bullets to chip away at poverty or improve national competitiveness, improving the ranks of teachers is part of the answer. That’s especially true for needy kids, who often get the weakest teachers. That should be the civil rights scandal of our time.
Sure. But wait:
The implication is that we need rigorous teacher evaluations, more pay for good teachers and more training and weeding-out of poor teachers.
It’s hard to see how anyone could jump to that conclusion. In the world of value-added research, Grady’s work would be judged not by whether she created a new reader who grew up to be a judge, but on the incremental progress of a large group of students that could be seen on a standardized test. If anything, the story suggests–contrary to what Kristof and supporters of value-added research like to claim–that figuring out what makes a great teacher isn’t necessarily going to be tied to test scores.
As Times education columnist Michael Winerip put it in his January 16 column:
The danger is that education policy gets driven by teaching methods that can be given a number.
I suspect that Mr. Noyes, my 11th grade Advance Placement American history teacher from 40 years ago, had a low value-added rating. As I recall, no one in our class got a top score of 5; I got a 3. There was no prepared curriculum aligned with the test: Mr. Noyes built the lessons. On any given topic, he would assign us several books that viewed history through different lenses–economics, politics, personality.
I have long ago forgotten the content of those lessons, but Mr. Noyes instilled in us something far more important: the understanding that history does not come from one book. While that idea has served me for a lifetime, I do not believe it is quantifiable.