I fell through the ceiling one afternoon. I was up there laying wire from one room to another. Although I had been very careful to stay on the rafters as I brought the wire to its destination, I got a little sloppy on the way back. Passing over our main hallway, I stumbled down onto the ceiling itself and crashed through. The result was a huge mess on the floor and a huge hole in the ceiling.
I stepped from the rubble ashamed and discouraged, awaiting the wrath of a vengeful wife. Au contraire. This was the break she'd been waiting for. After campaigning for years to get the “horrible popcorn ceiling” removed, we now had a reason to get the ceiling of her dreams. My stupidity was her bonanza. Twelve hundred dollars later, the hole was fixed and my wife was delighted.
The state of Washington, along with the rest of the country, has fallen through its own financial ceiling. We’re broke. Federal stimulus money has put off the inevitable for the past two years, but reality has now set in. In order to balance the state budget, class sizes will increase and salaries won’t.
Not unlike my wonderful wife, some people look at this disaster and see a golden opportunity. If we’re going to lay off teachers – and we are – why not lay off the ones who deserve it? Let’s keep the good teachers and fire the bad, regardless of seniority.
A recent story in the Seattle Times gets right to the point: “…student achievement after seniority-based layoffs would drop by an estimated 2.5 to 3.5 months of learning per student, when compared to laying off the least effective teachers.” This comes from a study by Dan Goldhaber and his colleagues, out of the University of Washington’s Center for Education data and Research. Using Value-Added Methodology, Goldhaber compared the 1,717 teachers who were given RiF notices last year with a comparable number of teachers who would be laid off if the decision was made purely on the basis of teacher effectiveness. As it turned out, almost all of 1,717 teachers kept their jobs, at least for now, but Goldhaber went to a lot of time and trouble figuring out that had we actually laid them off, we would be far worse off than had we laid off an equivalent number of lousy teachers.
There’s a lot to think about here.
First of all, Goldhaber’s conclusion seems to clash with an editorial he penned four months ago for the Seattle Times in response to the LA Times’ controversial decision to post VAM scores for their local teachers. The money quote: “There are reasons to be concerned that individual value-added estimates may be misleading indicators of true teacher performance. Teachers may not, for example, be fully responsible for the learning of all the students in their classes. Pullout programs, migration of new students into classrooms, and other ways in which instruction is specialized in schools, make the attribution of students to teachers complex.”
Goldhaber obviously acknowledges a validity issue when it comes to using VAM to evaluate teachers. At least he did back in August. This point is important. In my classroom, for example, the lowest readers are pulled out for remediation, up to an hour each day. Can I claim full credit for their progress? Furthermore, my district, like most, has moved decidedly in the direction of "scripted curriculum." To what extent can a teacher be held accountable, when he's essentially told what to say while teaching?
There’s also a fairness issue. VAM relies on math and reading scores. In my school we have 25 certificated employees. Collecting value-added data would only be possible for nine of those teachers. If lay-off decisions were based on VAM, how would we apply that fairly? Would the PE teacher get a pass simply because he doesn’t teach reading or math, regardless of how poorly he teaches PE?
Which brings me to the practical issue. The formula Goldhaber actually uses to compute teacher effectiveness, while controlling for all other relevant factors, looks like this:
Aijkst = ά Aiy(t-1) + Xit β + Cjt Y + λyj + Gt +φt + εijkst
That thing that looks like an upside-down Y stands for teacher effectiveness. The A in the front stands for student achievement. The other symbols stand for other things that affect student learning. If you showed this to an average teacher, who probably has little training in statistical analysis, and told her that this was how the state would decide whether or not she would keep her job, there would be trouble. Specifically, union trouble. For better or worse, this would take years, if not decades, to ever happen. By the time the state actually adopted a watered-down, union-approved version of this policy, we could very likely have a budget surplus, making the whole issue moot, and unnecessarily expensive.
(Fun Fact: I'm certainly no mathematician, but it would appear from this formula that teacher effectiveness technically isn't really a "factor" in student achievement; it's merely an addend.)
Finally, this would inevitably lead to unintended consequences. Specifically, we could see teachers migrating toward untested positions. Fourth grade teachers would move to second grade. English teachers would move to social studies. I can even envision principals assisting this migration: “We’re going to move you to first grade this year, Pat; and it trust me, it’s for your own good.” More subtly, even decent teachers like me would find it hard not to focus even more on "tested subjects" at the expense of science, social studies and art.
When it comes to saving their jobs, teachers are pretty good at gaming the system.
So where do I stand? Decidedly in the middle.
I appreciate the value of advanced statistical analysis. I get it. This stuff, if used correctly, in conjunction with careful classroom observation, could be extremely useful in improving student learning. More importantly, I understand the value of good teachers, and I hate that we have bad teachers.
But I'm forever the pragmatist. How do we get teachers to accept concepts like VAM as part of the new reality? As I pointed out above, there are a number of glitches to work out. Until we clear up the validity and fairness issues, there will be minimal buy-in from the teaching workforce. Minimal buy-in frequently translates into passive sabotage.
On the other hand, I see a lot of potential to use statistical analysis to evaluate programs, methodologies and curriculum. I found it frustrating, for example, that when my district chose a new literacy curriculum last year, we didn't do it on the basis of data; we looked at how it matched up with the state test.
And finally, a new ceiling is one thing. Someone's career is something else. And so is someone's education. Is it really right to capitalize on tough economic conditions by laying off underperforming teachers? Is it moral to use a financial disaster to rebuild our teaching force? Or immoral not to?
I honestly don’t know. What do you think?
I have to agree that it is hard to be evaluated by someone who is supposed to have more experience and is supposed to be a classroom leader, but isn’t.
This fall I was having trouble with a student. I went to one of my administrators and asked, almost begged for ideas to add to my toolbox for how to work with the girl. I had given her everything I had and knew how to give. The administrator had nothing to give me. Her only thought was she would talk to the girl.
Great ideas, Mike.
On another note, I was reading a report out of NYC about their use of VAM to rank teachers.
In summary: “The issue is when you try to take this down to the level of the individual teacher, you get very little information,” Dr. Corcoran said. The only rankings that people can put any stock in, he said, are those that are “consistently high or low,” but even those are imperfect.
“So if you have a teacher consistently in the top 10 percent,” he said, “the chances are she is doing something right, and a teacher in the bottom 10 percent needs some attention. Everything in between, you really know nothing.”
DrPezz, my concern with colleague-evaluations is that there is bound to be some tension if you work with the person you’re evaluating, especially if you give them a poor (but honest) evaluation. I like the idea of colleague observations, though, because we teach better when we’re observed.
Mike! I want you to come to Washington and be in charge! I love all of those ideas. How can we get that implemented? I think I’m going to start pushing our policy-makers in Washington to get the ball rolling on that. It’s brilliant.
We all want to keep the good teachers. What I wonder is, how do we know if they have “stick-to-it-tive-ness”? Half of all teachers quit within 5 years, because of low salaries and poor working conditions.
(http://www.washingtonpost.com/wp-dyn/content/article/2006/05/08/AR2006050801344.html)
I really love this blog… and that there seems to be several active and diverse contributors. A few thoughts:
In my system in Maryland we have what I would call a “partial” peer evaluative system. Admins still complete the evaluation, but master teachers serve 3 year stints outside the classroom providing feedback to both novice (1st year teachers) and tenured teachers rated unsatisfactory. I’d say they these master teachers make it into classrooms as little as once a month and as much as once every 8 school days depending on need of the teacher. The master teacher makes a recommendation but is completely independent from the admin. They are on the development side. The ultimate decision is made by a PAR (peer assisted and review) panel of teachers and principals. Good, but part of the problem is that tenured teachers rarely if ever make it into the system. Principals find documentation needed to place in the PAR system a pain. Teachers in PAR cannot make lateral transfers to other schools. And so it is under-utlized. I’d make some changes:
One, I’d make the teachers above a kind of evaluation team to be in charge of all evaluations- including firing. I like the 3 year rule- so that they’d have to return to the classroom. Admins would no longer handle evals- they would simply be in charge of the building- handling discipline and money and scheduling decisions. If they had issues with teachers they’d report to the teacher evaluators. In large schools, this would mean less administrators.
Inside the building, I’d like there to be a career path to become a different kind of master teacher- one focused not on administrative duties, but on instruction. I’d like this master teacher to be in the classroom maybe 60% of the time, but also have observation and staff development duties. THey would observe, support, and give feedback. Developing teachers would watch this teacher teach… in large schools, there would be several master teachers in a building. Like administrators they could report ineffective teachers to the evaluation team. The goal for every teacher would be to become a master teacher.
The first part, a roaming evaluation team would be expensive. THe 2nd part, placing master teachers in school building would just require a reshuffling of duties. Fewer admins, fewer team leaders or resource teachers. I was in a middle school once that must have spent 100 teacher hours planning for a 3 day outdoor edcuation event. A great event, but at a tremendous cost. Instruction takes a back seat.
Anyway- my big picture idea- is that we need people who are focused ONLY on instruction. If you’re pulled away from admin duties those admin duties tend to take priority. Admins HAVE to order supplies, or plan meetings, or make sure there is sub coverage. Instruction often takes a back seat. I’m willing to embrace any system that makes quality instruction the priority- that makes it a culture.
Maybe departments or staffs could vote for the so-called “master teachers” (or whatever moniker is preferred) to have a period a day to observe those in their departments/buildings. These teachers would teach one less period and use it to provide feedback to others. Possibly, these teachers could also invite others to do the same for them.
Maybe this type of system could be separate from evaluation or an informing piece. I’m not sure. Anyway, maybe this is a way to informally or amiably help all teachers.
Tom and I were on a panel a while about about defining the “Master Teacher” distinction on a potential future re-design of the salary model. During conversation, when we were “playing” (imagining a teacher evaluation model with no financial or time restrictions) we toyed with the idea of a traveling state-sponsored panel of teachers who would do these observations. Thus, there is no personal history between the assessor and the assessed, and there can be a certain degree of distant objectivity in a largely subjective evaluation.
It reminded me of when I worked for the Oregon FFA Association as a state officer and we did school visits to evaluate students’ FFA projects when they were candidates for their state degree. We travelled in pairs, and our sole job on that tour was to do site visits, evaluate the student and his/her project against a set of criteria, and then move on to the next site. Something similar (in a dream world) might work for teacher evaluation?
I am too vehement. I’m also a little intolerant and impatient when it comes to bad teachers taking paychecks.
I try to imagine myself being a peer evaluator, and I think I would need to be tempered by someone with a little more patience.
Maybe the same system assessors use for National Boards? The entries are read by two assessors, and if the evaluations are really different, a third assessor is brought it to tie-break.
What if classroom teachers took a year sabbatical to be evaluators?
I used to be against peer evaluations, because it seems like the best teachers don’t want to leave the classroom, but Mark, your comment changed my mind. You’re right. I’m thinking of my first principal, who taught P.E. for a year before she went into administration. Her suggestions for the classroom were totally useless, and her evaluations of us were mostly based on how obedient we were when she told us to meet during lunch, before school, and late into the afternoon after school.
In response to Kristin, I agree with pretty much everything, although perhaps not as vehemently. I particularly agree with the notion of principals stepping up and evaluating/firing. I am, however, more pragmatic in regards to change. I think we both want to get to the same place, but I’m willing to let the change happen gradually. Teacher buy-in IS important. The best evaluation plan in the world, without buy-in and support from the people who have to use it, will, in the end, be a failure. The second-best plan in the world, with teacher support, will be far more successful. And may eventually give way to the best plan. Change takes time.
Like the rest of you, I’m hoping for the best with the new state teacher evaluation plan. I’m also expecting the best, as I know some of the people involved in its creation, and they are smart people. I just pray that we’ll be able to afford it.
Peer evaluation? Maybe, but it depends on the peer.
One more thing; let’s not scapegoat the unions in regards to teacher evaluations. They actually have a legal obligation to represent members when their jobs are in jeopardy. It’s not unlike a public defender taking on the case of an obviously guilty suspect. I’ve seen the process from the inside; the unions are there to make sure the district follows due process; no more and no less. The main reason we have bad teachers in classroom is that administrators are either too lazy or too “nice” to follow the process of getting rid of them. And the process really isn’t that complicated. Believe it or not, the union people I’ve worked with are as committed to public education and good teaching as anyone who frequents this blog.
I like the peer-evaluation idea…provided that the teachers who do the evaluation somehow stay connected to a classroom of their own, in order to keep them in the “us” of the very real “us vs. them” dichotomy that evolves. In reality, it’s not so much the “us vs. them” that would concern me about having an administrator observe me… I actually value the feedback I’ve received from most of the administrators who have evaluated me.
I’m at a point in my career, though, where I’ve actually been in the classroom longer than all but one of my building administrators. I know that time doesn’t mean mastery or superior teaching, but when I’ve taught 20, and my supervisor has only taught 3 and then took some theory and observation sit-and-listen classes…and then the focus of the observation notes is on the number of kids using pens versus pencils when I ask them to take notes… there’s a problem. I’m hoping the new evaluation system in WA will prevent the kind of idiotic feedback that is the stuff of staff room legends.
Kristin, your point about the principals lack of action is why I believe we need to move to a purely peer evaluative model. We need to hire, montitor, professionally develop, and remove our own. Administrators can administrate, teachers can teach… I’m not against the model you suggest- however the “us vs. them” mentality would still persist- where unions serve the role as protectorate. I agree that this is why we have turned our collective heads to VAM- it helps administrators do their jobs- but at what cost? If someone would actually do their job- to evaluate- and could actually do it effectively and with teeth (where necessary), we wouldn’t be stuck here relying on tests of dubious validity, problematic reliability, and the associated negative externalities.
You know, more measurements are not necessarily a bad idea; however, value-added has been shown to be highly varying in the results provided.
Florida’s system was examined and from year to year teacher scores varied widely. Teachers would bounce from the top 30% to the bottom 30% in consecutive years. Did the teaching vary that greatly from one year to the next? Or, is the measurement one that will fluctuate greatly based on the students in the classroom?
Besides this, the focus on test scores is a huge detriment to education. It seems as though only quantifiable data is relevant now. Plus, as you stated, few subjects are actually tested. Where does this leave us as a fair measurement?
What few people are talking about–even though it may be the most significant change in Washington’s education system in the last 25 years–is the new evaluation system. This will change how teachers and principals are evaluated with a distinct focus on what happens in the classroom. The “check-off” will no longer be enough.
I read Goldhaber’s report, Tom, and was trying to craft a post but couldn’t get it right. I think you did.
It’s treacherous territory to try to design a better way to keep or let teachers go, because we can always identify the ways the new system is unfair. But the current system is as unfair as it gets. Unfair to excellent new educators, unfair to kids, unfair to families, unfair to taxpayers. It’s not acceptable, and it’s time we step up and fight to change it. We can no longer accept seniority as the best way to ensure job security.
We have to stop making decisions that suggest public education exists for the teachers. It doesn’t. we are here to do a job, and if we can’t do it well, we shouldn’t be employed.
So what do I think? I think we need to design a strenuous classroom-instruction based set of expectations. If a teacher teaches an academic subject, then it’s fair to include academic progress in the teacher’s expectations. Something like this sytem already exists in my district.
What needs to happen next, and what my district is also starting to do, is that someone needs to be hired to micro-manage the administrators to make sure they’re using the evaluation system and doing their job to move incompetent teachers out of the profession.
I think this whole thing rests more on the administrators than anyone is willing to admit. My principal is the only one who can fire the incompetent teachers in my building. Believe me, I’ve tried. He is the only one who can go through the process and move them out.
It is not that hard to have a better system for keeping and eliminating teachers. It’s been made harder than it needs to be because we are reluctant to fire the weak. They’re nice. We like them. They try hard and go to all the trainings. But the fact is, we wouldn’t hire an arborist who killed trees, and we protect teachers who damage kids or who waste their time. As union members we have to start making some noise. We have to design a better system for evaluating teachers, we have to accept that test scores will be a part of that for some of us, and we have to push administrators to do their job to ensure excellent teachers have jobs.
And you know what I think about teacher buy-in and passive resistance? I think it’s time teachers grow up and stop being spoiled brats. If an attorney acts like a jerk in the courtroom when trying a case, the judge will hold her in contempt. It’s not about buy-in. Why should society grant us that luxury? Why can’t there be certain job requirements, like teach children, know your content area, be able to demonstrate results? Why do teachers think we get to make all the rules?