By Kristin
As Brian revealed in his recent post, It looks like what happened in Los Angeles will inevitably happen in New York. Despite my frequent frustrations with ineffective teachers, administrators and unions, I have a big problem with newspapers like the Los Angeles Times or the New York Times who think it's their job to rate teachers.
Newpapers aren't in the business of teaching students or evaluating teachers; they're really in the business of selling advertising, so while they try to inform the public, they also tend to jump on the bandwagon of a hot topic in order to get readers. Rating teachers based on test scores is a hot topic right now, but it shouldn't be. If the public walked to a local school or logged on to a district's website, they'd be able to find all sorts of information. They could rate teachers on their own. Neither test scores nor a teacher's skill are kept secret, and it's misleading for newspapers to pretend they are.
What Brian points out is true – test scores are a complicated picture of what's happening in a classroom. I can't look through a microscope at a slide and diagnose an illness, and no one can look at a classroom's scores and rate a teacher unless you've also seen that teacher teach – a few times.
So, when officials like Natalie Ravitz say, "we believe that the public has a right to this information under the law," I wonder if she's aware that New Yorkers already have access to the important information, information more valuable than some research group's attempt to rate teachers it's never seen. States and school districts release school data that includes all sorts of things, from drop out rate to test scores to the experience and education of the teaching staff. Schools are public places, and as long as you sign in and aren't a registered sex offender, you can tour a school and get to know who teaches well and who teaches not so well.
Instead of eagerly scanning a list and chortling over some teacher rated as "less effective" than his peers, the public could tour a school. Newspapers are doing their readers a disservice to pretend rating a teacher can be done long distance, or that test scores tell the whole story.
The public can rate teachers on their own, for less than the cost of a New York Times, if they just get involved and take a look.
Here’s what I think teachers really want the public to keep in mind, and certainly the media, who does not teach: that there’s more to the problem than can be revealed by test scores, even, if as Bob says, top notch research groups are massaging the data.
Test kids, yes. Fire incompetent teachers and administrators, yes. Expect a teacher to teach regardless of the homelife, yes. BUT, you can’t use only a once-a-year test to evaluate a teacher.
Any number that measures a teacher’s impact should be the difference between a September test and a June test. Summer has to be a non-issue, because summers can have a big impact on kids. And the test has to be absolutely effective, with a sick child taking the test again so that the results are accurate.
If scores are used to rate a teacher, and the ratings are released to the public, then other data has to be included, like the student population and parental involvement. Why? Because like Mark, I spend about 20 minutes doing reading and math with my first grader every night. In fact, my daughter’s teacher has a classroom of children who have well-educated and involved parents. We’re helping her pull those kids along.
A teacher who has children who don’t do homework every night and on weekends has children who are going to move less from September to June. It’s not fair to publish my daughter’s teacher with a “highly effective” score, when part of that score is because of me. And it’s not fair to publish the name of a teacher who chooses to teach kids with uninvolved parents and chaotic homelives as “least effective.” And because there’s no ethical or even accurate way to release data about parental assistance, perhaps this kind of annual-growth data should not be release to anyone who doesn’t understand the student population.
I am not saying it’s not our job to teach. It is. I’m not saying a teacher can’t move any child forward a year or more in nine months, I really think we can. BUT, when newspapers release such neat and tidy ratings based on such a small part of a complicated story, it does more harm than good.
Emily – great point about the discussion, and hilarious point about people reading the newspaper.
I agree that the Times was really doing this to nudge the administrators, but I don’t see administrators on there with a little arrow down in the red zone.
What would the public reaction be if that had been the case? If the Times had said, “80% of Principal Miller’s staff isn’t effective, why is that?”
I think the reason we are so interested in critiquing the teachers is that teachers have contact with kids, but people seem to forget that administrators choose teachers and keep them in the classroom.
I have absolutely no authority to remove ineffective teachers from my building, even if their presence affects my ability to move my students one or two years ahead in 9 months.
Why do all of you think administrators have managed to dodge this whole examination?
First, I’d like to note that this entire conversation on this board would likely not be happening if the Times had not undergone this enterprising project.
Further, to say that newspapers should not offer information because “people will take it the wrong way” is just silly. The paper explained what value-added is as well as the fact that it is not the only measure to evaluate a teacher thoroughly in the articles and in FAQs on the website and in a web chat and … (http://projects.latimes.com/value-added/faq/#what_is_value_added)
If people “take it the wrong way” maybe we should be looking at why our nation’s teachers educated a society that can’t read a newspaper article and draw sound conclusions.
Oh wait, that’s kind of what the Times was doing, right? Questioning why administrators had never used readily-available data to help teachers and education improve…
Kudos to the LA Times for contracting with an objective world class organization to examine teacher performance. Rand essentially described and the Times reported what educators, real estate agents, parents, et al. discuss daily about teachers and schools. These people rate educators informally for a variety of reasons and have done so for decades that I’ve heard. The only thing new is the formalizing of the discussion. It remains unsettled what affect this formalization will have on measured student academic performance. I wonder if the reporting includes an inferred negative reinforcer, but I’m not sure for whom or what. Yes?
I forgot to mention in my previous post that about 20% of the teachers in the highest category in year one were in the lowest category the next. The same went for teachers in the lowest going to the highest. Forgot that. It was important to show the inconsistency.
Value-added scores are a very questionable method of teacher rating.
In Florida all of the teachers were looked at and rated using the value-added rating system. Only about 35% of the teachers in the highest category of the rating system were still in the highest category the next year. Only about 40% of those rated in the lowest category were in it again the next year. And people’s jobs are being determined using this system which shows that the ratings sway from year to year.
Another study revealed that an individual teacher’s scores could vary plus or minus 25%.
(This was part of a presentation to our district regarding the new state evaluation pilot.)
Well Mark, to be fair, the “value added” approach is supposed to measure growth, but if they are measuring a test the children took the previous spring to the test they kids took this spring, there’s a whole summer in there that’s unaccounted for.
Some families are able to really augment a child’s education during the summer, and some aren’t. If Ms. Wilson is being compared to a teacher down the hall whose students went to zoo school, or France, or who were taken to the library every day, then guess which teacher’s kids grew more from May to May? Not the teacher who teaches the disadvantaged kids.
I think the saddest thing about all of this is that if all administrators did their jobs to ensure teachers were being effective, and if all teachers were like Tom, who works hard on his own time to improve as a teacher, then the media wouldn’t feel they had to expose bad teachers.
If public schools did better, and we could and should, then the Times probably wouldn’t have felt the need to assess teachers on their own.
And, Fiona, I agree with you that they offered something to the conversation. Hopefully it will put administrators in the position of defending their teachers who were unfairly rated, or in the position of removing teachers who were fairly rated. I think many administrators have been allowed to slide along without really knowing what’s happening in classrooms, and one positive outcome of this scrutiny might be that they start paying better attention.
I teach math, but I have degrees in science. A huge problem in this debate about using test scores to rate teachers is that it is presented as scientific data, when it is clearly not. Science tries to unravel cause and effect by doing controlled experiments. In a complex problem the goal is to control everything except one variable and see what happens.
Releasing test scores is not scientific; it’s just raw data. The experiment would be to have a teacher work with a group of students and see what effect his instruction had on their progress. Then take the same group of students, give them a different teacher, teach them again and see if the results are different. Obviously that’s not possible. The first teacher had some effect, so the kids weren’t really the same for the second teacher.
But let’s pretend we could do this experiment, and we found that the first teacher did not add much value to these students. And we found that the second teacher added even less. So then we take a teacher who has been found to have added a lot of value to a different group of students and put her in the experimental classroom. She does the worst of all! What have we found out?
That the kids matter, and this is too complicated to pretend that it is science.
It is true that the Times may not have been claiming the last word on any teacher, but just as we teach our students, the intention of the message and the effect of the message are oftentimes two very different things. While the Times may not have intended it to be used to make a final judgment on any teacher, that is how it could easily be used. Data is often consumed without context even when context is offered–which is why it is dangerous.
I ran a reading assessment a few weeks ago and found that roughly two thirds of my students read below grade level–some testing as low as 3rd and 4th grade comprehension. I’ve crafted some interventions where I will be working in small groups to target the specific needs of my lowest readers. It would be dangerous of me to assume that I can work magic to get those kids reading at the 10th grade level by the end of the year, but I know I can increase their performance. So, let’s say they read at the 8th grade level consistently by the end of this year…I’d say my efforts were a success: but not in the eyes of those who’d look only at the snapshot data in June and who would see that so many of my students were reading below grade level…regardless of their progress. That’s the kind of context that is really needed in order to meaningfully assess a teacher’s effectiveness.
I have always enjoyed working with kids who struggle, and I believe that I am good at what I do. However, if you compare my students one-shot test scores to other groups, my kids underperform. That underperformance doesn’t consider how far they’ve come. If my name were published in the paper with even the implication that this meant I was ineffective, it would be the beginning of the end for me. There is no way I’d put myself through that kind of decontextualized scrutiny by choice. I know Oprah and the rest of the country want a bunch of martyr Supermen (you know, the ones they’ve been waiting for), but the fact is, we’re going to have a hard time keeping good teachers with the students in the most need if this is the way we plan to treat them.
Publishing test scores in the newspaper isn’t “accountability.” Accountability would be that newspaper saying to the teacher: “prove you’ve done your job; show us how you’ve facilitated GROWTH.” A single test score will never truly function as “accountability” in my view. If we get to the point where my job hinges on my test scores, I’ll be looking for a new job since I’m not going to paint a target on my back just because I choose to work with kids who have great ground to make up.
I agree, but I think the Times did provide some context, though perhaps not enough. It would have been better, perhaps if there had been clearer links to information about why, for example, it doesn’t make sense to compare the performances of teachers who work with vastly different populations. And it would have also been nice if there had been some assurance that those kids who weren’t an entire year in any particular classroom were thrown out of the sample. They should have been.
But still, I think the newspaper did an admirable job of squeezing something meaningful out of data which in its raw, publicly available form, didn’t have much useful to offer anybody. It’s a far cry, say, from the Fraser Institute rankings, which are just silly.
And I think that public investigations like the Times one offer something valuable to the conversation. They’re not claiming the ratings are the last word on any particular teacher.
Fiona, thanks for commenting! Here’s the tricky part about what the Times did in LA: a teacher might have been rated “less effective,” but there is no data other than the rating. In fact, I just searched for a teacher with the last name of Wilson (random choice) in the LA Times ratings. You can see her score here: http://projects.latimes.com/value-added/teacher/marquetia-shero-wilson/
She is not only “less effective,” she is pretty much “least effective.” That tells you something more about the teacher than the school’s scores, you’re right, but it doesn’t tell you as much as if you observed the teacher in action.
True, Ms. Wilson’s rating might be because she does little in class, doesn’t know her material, and can’t teach, which you’d learn in five seconds watching her.
Or, her rating might be because she chooses to work with kids who have high, high transiency. I’ve had students like that, who are here for a month, go to another school for a few months while they try life with dad, then return to me for testing because dad didn’t work out and they’re back with mom. Or they might be because she works with kids who have high truancy.
We just don’t know, but they could tell us. They could include that data in their value-added rating.
The problem with the kind of “you’re this good” rating system that the Times released is that it doesn’t tell enough of the story.
I am for testing and I am for teachers having an impact on kids, but when it comes to rating teachers, there has to be more information taken into account.
I think what really needs to happen is that this kind of rating system is part of a bigger evaluation – one with some teeth. If Ms. Wilson really is pretty bad, then six years’ of data should be used alongside classroom observations to get her out of the profession, and someone should be asking her administrator if he or she knows what Ms. Wilson is like in the classroom.
In Seattle at least, the only test data that a parent can immediately look up is the grade-by-grade standardized test results every year. Each figure represents a different group of kids and teachers, and it’s impossible to learn much, other than the impact of social class on test scores. There is a big difference between this and the kind of information available on, say the Los Angeles Times posting. The LA Times looked at six years of data for a group of teachers, and made the rating based on how much progress students test scores showed after a year with the teacher.
Here’s the Times’ explanation:
“A teacher’s value-added rating is based on his or her students’ progress on the California Standards Tests for English and math. The difference between a student’s expected growth and actual performance is the “value” a teacher added or subtracted during the year. A school’s value-added rating is based on the performance of all students tested there during that period.”
The LA times doesn’t pretend the numbers catch everything.
“…value-added measures do not capture everything that goes into making a good teacher or school..”, but I think they are providing a valuable service. If my child’s teacher did badly in such a rating, I’d want to know why. There may well be a good reason, but I’d still want to know.