A new research paper by a team of economists got a lot of pretty favorable press because it appears to deliver results that would seem to confirm what many in the media believe about American schools: If you could just use standardized test scores to weed out underperforming teachers, you would see serious improvement in school achievement.
Media coverage often glosses over the core problem here, which is how you measure teacher performance in the first place. The “value-added” research that is touted by many pundits–using test scores to determine a teacher’s effectiveness–is controversial in large part because critics don’t think it does what its supporters say it does (not to mention that dramatic swings in such scores from year to year, which can make a teacher “great” one year and below average the next). These are rather important criticisms that value-added boosters should engage.
Or they can be New York Times columnist Nicholas Kristof. In Kristof’s first column on the research (1/12/12), he cheered the study’s suggestion that good teachers boost student incomes:
Each of the students will go on as an adult to earn, on average, $25,000 more over a lifetime–or about $700,000 in gains for an average size class–all attributable to that ace teacher back in the fourth grade. That’s right: A great teacher is worth hundreds of thousands of dollars to each year’s students, just in the extra income they will earn.
There have been several interesting critiques written of this, which was written by Harvard’s Raj Chetty and John Friedman and Columbia’s Jonah Rockoff, but has not yet been peer-reviewed or published. At the United Federation of Teachers blog (1/8/12), Leo Casey argues that value-added research
assumes that standardized exams are accurate, reliable and robust measures of actual student learning, a necessary assumption if one is to use them as a measure of teacher performance. It is tautological to claim that an analysis proves what it assumes, especially when that assumption is precisely what is contested in the public debate over standardized tests and value added measures.
Casey goes on to note that the singling out of future earnings–which featured so prominently in the coverage of the study–is also problematic. He cites another critic, education writer and scholar Sherman Dorn, who wrote:
If you want to generalize this claim beyond the data used for the study–associating the group effect scores with teacher quality more generally, making claims about lifetime income, or extrapolating to policy questions–you are making assumptions beyond what the data support.
These are some of the many criticisms of the study. But Kristof’s follow-up column (1/22/12) skipped any serious discussion in favor of this caricature:
After I wrote about the study, skeptics of school reform wrote me to say: Sure, a great teacher can make a difference in the right setting, but not with troubled, surly kids in a high-poverty environment.
Who is arguing that poor, “surly” kids can’t be reached by good teachers?
Kristof then goes on to find a living, breathing rebuttal to an argument no one is making. “Olly Neal was a poor black kid with an attitude,” Kristof tells readers. His life turned around when his teacher, Mildred Grady, started buying books she thought he might enjoy and placing them in the library. That changed Neal’s life–he “caught the book bug,” went to college and eventually became a judge. And thus, Kristof argues:
To me, the lesson is that while there are no silver bullets to chip away at poverty or improve national competitiveness, improving the ranks of teachers is part of the answer. That’s especially true for needy kids, who often get the weakest teachers. That should be the civil rights scandal of our time.
Sure. But wait:
The implication is that we need rigorous teacher evaluations, more pay for good teachers and more training and weeding-out of poor teachers.
It’s hard to see how anyone could jump to that conclusion. In the world of value-added research, Grady’s work would be judged not by whether she created a new reader who grew up to be a judge, but on the incremental progress of a large group of students that could be seen on a standardized test. If anything, the story suggests–contrary to what Kristof and supporters of value-added research like to claim–that figuring out what makes a great teacher isn’t necessarily going to be tied to test scores.
As Times education columnist Michael Winerip put it in his January 16 column:
The danger is that education policy gets driven by teaching methods that can be given a number.
I suspect that Mr. Noyes, my 11th grade Advance Placement American history teacher from 40 years ago, had a low value-added rating. As I recall, no one in our class got a top score of 5; I got a 3. There was no prepared curriculum aligned with the test: Mr. Noyes built the lessons. On any given topic, he would assign us several books that viewed history through different lenses–economics, politics, personality.
I have long ago forgotten the content of those lessons, but Mr. Noyes instilled in us something far more important: the understanding that history does not come from one book. While that idea has served me for a lifetime, I do not believe it is quantifiable.
The unmitigated gall is palpable when those who don’t give a good goddamn for poor children use them as props for promoting policies designed to keep them poor.
You could call it a standardized test of hypocrisy among media elites, whose kids will never have to worry about the effects of their parents’ propaganda on their educational opportunities, will they?
An EXCELLENT summary of this whole subject can be found in the 1995 book “The Manufactured Crisis – Myths, Fraud, and the Attack on America’s Public Schools” by D.Berliner & B.Biddle, two professors. It may not be readily available, but they knock down all this stupid crap that the right-winger/anti-teacher-union faction has been using as a distraction /’blame-the-victim’ ruse for these past two decades.
If legislators truly believed that standardized testing (a numerical weighing of value) improves educating, and they wanted to improve the lamentable state of legislating, we would have a set (or two) of standardized tests to measure legislative outcomes. (Spare me the myth that elections are fair tests of value.)
we also could improve the quality of beef by weighing the cow…..
I question whether the experts are really interested in improving education–I think they are more interested in making money from it with privatization, online learning, destroying unions so they can get cheaper teachers, and making sure that only the elites get a really good education. Even the sincere ones seem to have lost the distinction between “education” and “training.” the school improvement they advocate is all geared toward training and education is devalued–if acknowledged at all.
If the reformers were really interested in improving education, why don’t they do the obvious? See what the best practices are in all the countries that are cleaning our clocks, and see what we do that no other country does. That research has been done and published, but is being studiously ignored: See http://mcgraw-hillresearchfoundation.org/2011/03/16/mhrf-pisa-paper/
I suspect Nick Kristoff’s example could be better explained by the fact that the surly kid got some individual attntion from his teacher. The teacher found out what would turn the surly kid on and followed through to see that it got done. Each child needs to have teachers who care enough to find out what turns them on.
I also question what increasing income by $25,000 over a lifetime really means. Is a lifetime of work 40 years, maybe? In that case we’re talking $650 a year more for each student–$12.50 a week. The amount is so minuscule that I can’t believe it holds any significance at all. Couldn’t Kristoff have noticed that at least?
Teaching to the who, what, when, where, why, and how of any subject and demonstrating relationshps and applications to real life works for me better that teaching to the test.
Love of learning and encouraging curiosity, plus looking at a problem from more than one direction is what all of my best teachers gifted to me. It didn’t hurt either that both of my parents read a lot, and also treated my thoughts and questions as important. I was a lucky kid.
The biggest problem with American education is that the nation does not seem to value education,nor does the nation value looking at a problem from more than one viewpoint. Congress is certainly a good example of that. Taking that through line of ideas into possible consequences seems to lose out too.
Looking at income does not give a clear idea either.After all, Wall St.certainly gives out high employee income, but sadly they have left behind such poor outcomes for the majority of Americans and the nation too.
Facts are important for little kids because that gives them a sense of accomplishment and a way to view i human natiure through past history; a timeline of human greatness and also the truly sad. At a certain point though, the students have to learn to do something with those facts, and learn that the world is more than statistics, which can be analyzed in so many different ways. Bubble tests measure facts more than they measure thinking. If a student knows that the Magna Carta came about in 1215, but doesn’t understand what that document gave to humanity, then what’s the point?
I would like to give out a challenge to all of those who think that teachers are the problem. Spend a month in a classroom and report back on what you find. I think those reporters would find themselves exhausted and soon looking for a way to pull that fire alarm so that they could exit as quickly as possible. As a reminder to those reporters who don’t understand, I think it’s best to give them with this thought by paraphrasing GB Shaw: “Those who can DO, those who can’t PREACH!”
Standardized tests are a very poor measure of student learning, but they are used mostly for political purposes, as any teacher knows. To really measure the value of a teacher’s work, one has to look longitudinally at who the kids were than came through the door–public schools take them all–and what becomes of them later. I am happy to say that this summer I will picnic once again with a group of “kids” I had in junior high school 1968-1970. None of those kids went to jail, none secumed to drug addition, all are gainfully employed. Most seem happy and normal. I love them and they love me. We went though some horrible times together in that era, with National Guard troops around us on their way to OSU and Kent State. Of course, I don’t know how much I had to do with their survival and successes, but they tell me our class meant a lot to them. We need other such measures.
Jack Burgess, former Executive Director, Columbus Education Assn.
You can’t make a pig fat by weighing it all the time. Good teaching is the answer and not continued time lost on test taking.
The drive to evaluate teachers according to the test has a lot more to do with disciplining children for the demands of an ever-more business-dominated society and deprofessionalizing teachers than it does with education.
One only has to take a look at the formula for Value Added Modeling to see it is a farce. You can see actual formula in the URL below, in an article by Michael Winerip of the New York Times. I find it appalling how many people think they know how schools should be run, but have never taught. Basically, No Child Left Behind says 100% of children must meet standardized testing goals or a school had failed (by 2014). Who else works in an environment that demands perfection based on criteria that few seem to question. The assumption is the tests are valid, all children can pass them, and that failure is an indicator of poor teaching. Not of these are true, but then I never thought the current reforms were meant to save our schools. The goal is to privatize them, just as business wants to privatize other public institutions.
http://www.nytimes.com/2011/03/07/education/07winerip.html?_r=1
Did the economists who researched the issue of how to improving school achievement ever teach in a classroom? How can people who’ve never stood before a classroom have the answers? Why not ask the teachers?
From my perspective, there are three components in the learning process. A good teacher is very important, of course. What about the learner, however? Is the learner not responsible for his own learning to some degree? To say that he’s not, is to say that the learner plays no role in his own learning experience which makes no sense.
The last component is the home. Parents who are involved, supportive and interested play an important role in the learning process. Many parents are so busy with their jobs that they don’t have the time to devote to the learning process.
It appears that only one component is discussed–the teacher– and every expectation for success is put on the teacher’s back. It’s like a three-legged stool, with only one leg standing.
From what we see in politics, I’d say higher income correlates with worse education, not better.
Student tests used to evaluate teachers need to include a student profile in order to qualify the results.
Sample profile questions:
Do you enjoy this class?
Are you interested in the subject matter of this class?
Do you feel you have good support at home for your studies?
Do disruptive students interfere with the teacher’s efforts in this class?
Each student’s self-profile, reported anonymously with the student’s test results, should be evaluated by an independent party that has interviewed the student where resources are available.
Those interested in the problems of educating the youth of the U.S. should see John Burris’s editorial in the 13 January, 2012, issue of Science, the journal of the AAAS. My own two-cents worth is that we need to pay serious attention to children from birth until graduation from high school. We all lose when poverty or neglect dampens the curiosity of a child. Unmeasurable, but undoubtedly vast, amounts of talent are destroyed or diluted by poverty and neglect.
Dewey: Yes, I think it’s pretty well established that socio-economic level and educational achievement often go hand-in-hand.
Great points, Jared.
Elaine hold onto your hat…….I agree with what you wrote :)
From my personal experience,I have gone through strict schools with Notre Dame nuns,Franciscan brothers and profs that run the gamut.The gamut being what I consider to be great teachers and horrible teachers.Esteemed Harvard has some really crappy profs(in my view)And yet one of the greatest I had was a class I took one summer in a community college.He was brilliant and really engaged us.I think the problem today is locking in all your child’s future- to his teaching staff.This is only one aspect.I think the majority of the things that allows you to be happy(that may mean wealthy….. or not)still happen at home.School is not on the whole a replacement for that.
I know a teacher now that works till 8 many nights.She probably makes 70k.Im sure she is fantastic.I know a certain Ivy L prof that make near 300K who goes in at 12……does his E mail for an hour or two……teaches his class and goes home.Four days a week!
Should teachers be judged?Of course.No other industry goes without some standard.How is the nut.Do I think the unions are helping or hurting?Well where I live we have 1500 tenured teachers in one accounting.One was fired last year.One!That is a reveling stat
One of our areas is fighting paying anything toward healthcare.Will privatization help or hurt?Again that is a tough nut.one thing I have always thought was wrong is(sorry my conservative friends)is the tax base of an area paying for that areas schools.I think it should be evenly distributed over a wider(state?)level.I do see the other side but….
I said there were three components to the learning process. The teacher was one of those three. However, the student plays a role, too. When students come to class prepared, when they study for tests, complete homework assignments, pay attention, work hard, they contribute to their success. In short, it’s not just the teacher. It’s the student, too, who plays a role in a successful learning experience.
The family has a role as well. Families who are make it clear to their kids that school comes first, who are involved and supportive also help create a successful learning experience.
Yes absolutely.
Based on my 15 years’ experience in leading two high performing high schools which served a significant number of minority and free/reduced lunch students, the best secondary education available is through the International Baccalaureate Programme. Part of the program is the evaluative feedback given teachers on their materials, the ways they assess students in class, their preparation of students for both the teachers’ own assessments (“tests”), and the students’ performances on the standardized IB essays. Although such a system is more expensive than the more simplistic testing regimens proposed by States in the U.S., it would provide a more in depth and reliable approach to evaluating teachers than the “value-added” systems discussed here.