By Kristin
Seattle has rolled out "Student Growth Ratings." Some teachers are devastated, some confused, and the vast majority are unaffected. Next year all 4-8 reading / math teachers and all 9th grade algebra teachers will receive SGRs. These teachers are called, "Teachers of Tested Subjects." Despite the HSPE being THE big test students need to pass before graduation, 10th grade LA/Math teachers are not considered "Teachers of Tested Subjects." Last week some teachers were told they had "low," "typical," or "high" student growth. Watch this overview video if you are curious.
It's now state law that student test results be used in teacher and principal evaluations. How that information is acquired and used, though, is up to local bargaining. My district has packaged that data by taking a teacher's roster and measuring each student's growth against two different comparison groups in order to get two scores, a Student Growth Percentage and a Value Added Measure, using spring MSP and MAP scores.
First, for the Student Growth Percentage, the student's growth between one spring's assessment (MAP or MSP) and the next year's is held up against that child's "comparison group," a group made up of the other children who shared the first spring's score. If a child improved more than 92% of her comparison group, her SGP is 92. That's pretty good.
The second measure that's used is a Value Added Measure. You can watch the video for it here, but it's a little like watching someone switch cups around so you don't know where the pea is. Value Added takes into consideration a student's comparison group regarding Special Ed services, being an English Language Learner, and Free and Reduced Lunch eligibility. The district has made a prediction of a child's growth based on these factors, and then compared the child's growth to others in his/her comparison group. It is possible for a child's predicted growth to be a DROP IN SCORE. While I appreciate that someone out there understands that a child who travels to France in July with a backpack full of Barnes and Noble's latest is likely to improve more than a child who spoke Somali all summer and babysat her younger siblings, I'm not comfortable being told my district has high expectations for every child except poor, learning disabled English Language Learners.
These two numbers are added up and averaged, magically turned into a number between 1-100 (that's explained on a mythical 5th video referenced but, so far, impossible to find), and teachers are placed in a "low," "typical" or "high" category.
In my opinion this is not being done well. First, assigning teachers a rating that has any margin of error at all - and this one has a margin of error of 10-15% – diminishes the worth of using student data at all. Is it credible to call a teacher "low" if his student growth rating is 34 ("Low") but there might be a 15% margin of error, and the score is actually 39.1 ("Typical")?
Second, I'm not comfortable with a teacher's ability to move kids being measured with a test like the MAP, which is an assessment teachers are absolutely unable to prepare children for. There are no released items for the MAP. There is no way to prepare kids, except to hope that teaching to common core standards will do the trick. I stood behind a student as he grappled with the question, "How does the use of a malapropism affect the meaning of John F. Kennedy's speech in the text above?" I could only hope that my student could wing it with his understanding of "mal" and his ability to use the process of elimination to pick the right answer.
Third, my daughters' test scores are pretty far down my list of priorities for what they get out of school. My younger daughter's MAP score was in the toilet last spring. Do I want her teacher to sit my daughter in a chair and drill her until her math scores climb to an acceptable level? Hardly. I'd prefer my daughter to be learning about what it is to be a human in this world, about science, and social skills, and being an effective problem solver and independent learner. I measure her teacher's effectiveness by how much my daughter looks forward to Monday mornings, and I don't think I'm alone in that.
Finally, I'm uncomfortable with where all of this is leading. We can't go back, and I don't want to go back, to teachers being secure in their jobs even when their children learned nothing. Looking at how much students grow under a teacher's care is appropriate, but we must do it right. A margin of error is not acceptable. Holding a very small percentage of teachers accountable and giving the others a career off is not acceptable. Measuring a teacher's effectiveness with assessments that mean everything to the teacher but nothing to the student is not acceptable. The End of Course exams, used to determine whether or not a child moves on, are much more credible measures of a child's ability.
Already I'm hearing colleagues consider private school, think of ways out of teaching "tested" subjects or the same subject two years in a row, and I'm hearing frustration and bitterness. If the point of this exercise is to reward excellent teachers and encourage them to teach challenging kids, the mission will fail if those very teachers are demoralized, targeted, and hopeless. The bottom line is that Student Growth Measures don't help students as much as a more thoughtful, meaningful measurement of growth would.
As the voice says on Seattle's videos, "Because of the challenges associated with quantifying the contribution of a teacher to student learning, the evaluator is strongly cautioned against making assumptions about the teacher's instructional effectiveness." If the evaluator can't make assumptions based on this data, then what's the point?
It’s really helpful to hear how other districts are meeting the student growth measurement requirement.
I heard Seattle does have a September/June assessment being piloted. I’m going to try to find out more.
Mark, these growth ratings are supposed to affect a teacher’s “career ladder” opportunity. This means if you receive a “high student growth rating” you can apply to be a demonstration teacher, or star mentor (a program to help new teachers) or other leadership opportunities. Three years ago I “opted in” to the new, rigorous teacher evaluation system because the carrot of “career ladder opportunities” was dangled so attractively before me, but even though I met all the requirements I wasn’t eligible after all, because only certain buildings were. Now I believe all the extra money that paid the salary of those teacher leaders is going away. I don’t have a lot of faith that there will be any real meaningful, equitable career ladder opportunities for Seattle teachers.
If you receive a low rating, you have some consequences. You’re put back into the most rigorous evaluation – two formal observations a year with pre and post conferences, and all the paperwork and rubric work those entail.
It bothers me that only some teachers – teachers of grades 4-8 math and reading, and 9th grade algebra teachers – are intensely managed in this way. A really awful history teacher, as I have learned from personal experience, can undo the work of most of a school’s staff. Why doesn’t my district expect high school teachers to move kids? Why doesn’t my district expect science teachers to move kids?
I do not know.
I just re-read your first link… the closing paragraph says that teachers with higher rankings will qualify for a promotion? To what? None of this makes any sense whatsoever.
Be very, very wary of valued-added measures, and I would advocate bargaining it right out.
VAM rank teachers; they do not reliably or validly assess teacher effectiveness or student growth.
My district, in my opinion, has wisely chosen to show student growth through the standards in an individual course; in fact, we recommend against any state test. The most accurate and reliable assessments for showing student growth are classroom-based.
Also, the state law does not specify how long between points in time a measurement must occur. Could the beginning of a period, a week, or a unit be used? Absolutely, and probably should.
*oops, in my rant I meant “it does not demand that teacher A demonstrate MORE student growth than TEACHER B” not “student B.” I guess I missed that “close editing” learning target 🙂
Heinous.
This is what the law actually says: (RCW 28A.405.100 2f)
“Student growth data that is relevant to the teacher and subject matter must be a factor in the evaluation process and must be based on multiple measures that can include classroom-based, school-based, district-based, and state-based tools. Student growth data elements may include the teacher’s performance as a member of a grade-level, subject matter, or other instructional team within a school when the use of this data is relevant and appropriate. Student growth data elements may also include the teacher’s performance as a member of the overall instructional team of a school when use of this data is relevant and appropriate. As used in this subsection, ‘student growth’ means the change in student achievement between two points in time.”
Unfortunately, the press sees “assessment” and “data” and says “test scores,” even though the word “test” is deliberately avoided in favor of “assessment.”
What Seattle is doing is a waste of time–it simply doesn’t align with what the law says, and therefore, if it is part of a teacher’s evaluation, it will be challenged. It is an attempt to use numbers for one thing when they are intended for another.
A first issue: “multiple measures.” This doesn’t mean state test from one year and state test from the next. This fails the “multiple measures” test, even if one can argue that it passes the “two points in time” test (which I suggest it also fails, since two points in time, in my non-lawyer opinion, means two points in time during the teacher’s contact with the student). Multiple measures means multiple means of assessing/determining a student’s achievement toward a learning goal/standard.
The biggest reason that this data mess in Seattle will fail: the law demands that a teacher demonstrate student growth. It does not demand that teacher A demonstrate MORE student growth than student B. The whole (state) system is premised upon evaluating teachers against a standard (hence, the instructional frameworks and the rubrics) not against one another. If teacher A’s student growth data is not convincing, then it is up to the evaluator to ascertain that and evaluate appropriately.
Further… I don’t know what other frameworks look like, but we’re a Marzano district, and our student growth rubrics specifically talk about a teacher’s ability to select valid assessments that are aligned to learning goals based on student needs, and then whether growth can be demonstrated. It puts a ton of responsibility on the teacher to craft/select assessments that measure students in a valid and meaningful way.
For example, in the Marzano rubric, component 3.1 (student growth data)… to earn a proficient rating, the teacher “Establishes appropriate student growth goal(s) for subgroups of students not reaching full learning potential. Goal(s) identify multiple, high-quality sources of data to monitor, adjust, and evaluate achievement of goal(s).” Then, 3.2 (sg): “Multiple sources of growth or achievement data from at least two points in time show clear evidence of growth for most students.” For component 6 (sg), substitute “whole classroom” for “subgroups” above.
Component 8 is this (at proficient level, again the sg scale): “Consistently and actively collaborates with other grade, school, or district team members to establish goal(s), to develop and implement common, high- quality measures, and to monitor growth and achievement during the year.”
I don’t teach in Seattle, so I don’t know what the frameworks say up there. Where I teach, though, (1) the district so far has been smart enough not to dive into the state-test-score minefield, and (2) our framework language related to the student growth data elements seems to preclude the kind of teacher-comparison-rankings that are happening in Seattle. Thank goodness.
Your discomfort with all of this is completely justified, and I’d be doing everything I could in your place to undermine faith in the system – because it’s indefensible, from a scientific standpoint. There’s no reasonable claim that this system is about improving teaching because the scores arrive too late to be useful, the ratings and adjustments are arbitrary, and the test is not instructionally sensitive. I’m not sure if I agree we can’t go back. I envision lawsuits. http://accomplishedcaliforniateachers.wordpress.com/2011/11/02/vam-on-trial/