By Tom
When I was younger I used to work in a swimming pool. Every day a guy named Nick would emerge from the boiler room and test the water. He would hang a thermometer in the pool and give it time to become accurate. In the meantime, he'd fill a test tube with water and check the ph. Then he'd collect his things, walk back to the boiler room, enter the data into a log and take whatever actions the results dictated.
Water is simple. It reacts consistently to external forces. And our swimming pool had an efficient circulation system. Water came in through valves in the bottom and it left through drains in the gutter that lined the side of the pool. There was also upwards of 100 people in the pool at any one time, swimming and splashing around, stirring things up. It was therefore reasonable to assume that testing one area of the pool would yield a result that was fairly representative of the entire pool. If the ph in Nick's test tube was off, then there was something wrong with the water in the whole pool.
But just to be sure, Nick would double-check whenever he got a weird reading. He would go around and test other parts of the pool, because wanted to make sure there really was something amiss before dumping an extra shot of chlorine into the pool.
Nick understood testing. He knew its value as well as its limitations.
I'm not so sure we do.
I just received the the reading scores from the state test that my students took last year. They came last week. By and large, most of my students performed well. If we had a merit pay system in place in my district, I'd probably be able to take my family to Disneyland next summer. It was good news.
And as I looked over the results, I saw confirmation of what I observed all last year. The kids who read well day in and day out performed well on the test. The kids who struggled throughout the year performed poorly on the test.
With one exception.
This One Girl had a low score, even though she was far and away the best reader in my class. She reads fluently; the last time I measured her, she read 187 words per minute, which is really fast for a third grader. She also has great comprehension skills. She can recall the main idea with supporting details, she can produce a summary of a story, she understands the authors' purpose and can tell about the mood of a given story. She is, in short, an accomplished reader.
Yet her score on the state test was "below standard."
I'm not concerned about myself. I know This One Girl can read, and I know that I helped get her there. I did my job. And the fact that 24 of my 25 students had test results that confirmed what I saw all year in the classroom tells me that I know what I'm doing with regards to assessing and reporting to parents. In other words, as far as I'm concerned, the state test was 96% accurate in terms of telling me how well I know how to execute a sound literacy program. That's not bad at all. But that's not what this post is about.
It's about This One GIrl. The good reader who got a low score on her test. I'm not sure what happened. Maybe she skipped a few pages by accident. Maybe she was reading too much into the questions and not answering them with her common sense. Maybe she was freaked out by the whole experience.
I don't know what happened. But I know what will happen. She'll get her scores back and think she's a bad reader. Her parents will see her scores and think she's a bad reader. Then they'll make her read more at home, which isn't a bad thing, but it'll likely come at the expense of her dancing lessons, which she really loves. And her fourth grade teacher will see her scores and think she's a bad reader. She'll put her in with the other low readers who need extra support. She help her choose easier books to read. She'll ask her simpler comprehension questions. At least until she figures out that This One Girl's actually a really good reader. Things will eventually get sorted out, but in the meantime she will, for all intents and purposes, become a bad reader.
Now I know that testing entails a certain, acceptable level of error. It's inevitable. Just like with the swimming pool, the readings can sometimes get weird. And then it's up the testers to figure out why.
But that's the problem with our state's tests. We aren't allowed to know why. The rules dictate that teachers cannot look at the tests before, during or after their students are completing them. We can't see the questions, we can't find out which questions a lot of our kids are having trouble on, and we can't look through the tests after they're turned in to see if someone skipped a bunch of questions by accident. We're not allowed to.
So when I talk to This One Girl and her parents and her teacher, I won't be able to explain what happened. Because I won't know. I can direct the parents to the website that has directions for viewing their daughter's test, but that's a huge hassle. They have to make an appointment with the school district and look at the test in the presence of a district administrator who won't know the first thing about their daughter or the test she took.
Tests are important. They're useful as a check to see if students, teachers and schools are on the right track. But we need to remind ourselves once in awhile that we're measuring people. Little kids and big kids. And we need to remember that tests, like the people who write them and the people who take them, sometimes make mistakes. And we need to at least be able to explain how those mistakes happened.
Like with This One Girl.
Actually, Jason, we do have our test results disaggregated by sub-skills out in Washington. (Some of us even have indoor plumbing!) But it’s still impossible to know exactly what went wrong with this girl’s test without actually looking at the original document, which I’ll never be able to do. More to the point, though, a test that’s 96% accurate in regards to telling whether a child can or can’t read is good enough for me, good enough for the district and good enough for the state. Given the current high-stakes attitude surrounding testing, however, it’s not good enough for the one kid per class for whom we get it wrong. Consider this analogy: You might decide to buy a deodorant that’s 96% effective. That’s because the stakes (body odor) aren’t that high. But you wouldn’t consider buying a car with brakes that were 96% effective, would you? No, because the stakes are too high. Now, I’m not saying testing has become a life or death issue, and I certainly don’t favor abandoning testing altogether. All I’m saying is that when we ramp up the stakes, we need to be prepared to deal professionally and sensitively with the kids for whom these tests are not accurate. And one way to start is to simply let their teachers look through the completed tests, just as they do the rest of the work their students do in school.
Mark’s right; if I had seen the original document I could have had a timely conversation with everyone involved and avoided this whole mess.
And think of how much more useful it would have been to know this before This One Girl left your classroom for the year. You could have had timely conversation with her parents, maybe even her next year’s teacher.
As disconcerting: her test score, according to many, reflects more about your capacity as a teacher than it does about whatever might have happened on test day. If you had just done your job better, Tom, she would have passed. I don’t believe, that, of course.
Washington state doesn’t give question-level data back to teachers? That’s ridiculous. I understand not releasing the questions, which inevitably leads to coaching and inflation. But if Washington isn’t at least presenting data on a question-by-question level with some minimal question-type classification, well, they’re just being lazy and irresponsible.
Also, any district that makes choices about instruction based solely on end-of-year exams is not doing their job.