By Tom
A few years ago I was giving my third graders their annual standardized test. This was the reading assessment, and Rachel had her hand up. I asked her what she needed. She wanted to know what a “selection” was. She was stuck on a question that asked her to pick the correct main idea for the “selection” she had just read. Now, you and I know that “selection” is the generic term for any form of text, whether it’s a poem, an essay, an article or a story. But when you’re in third grade, the generic term is “story.” Rachel was a good reader, and if I had told her that the question was asking her to pick the main idea of the “story,” she would have been just fine. The question was clearly directed at her ability to find the main idea, not her understanding of the term ”selection.” Nevertheless, I wasn’t supposed to explain it to her.
I was conflicted. Should I define the word, thus enabling the test to actually measure what it was designed to measure and enable Rachel to demonstrate a skill that she actually had? Or should follow the letter of the law and do what I was told to do during the 20-minute Proctoring Workshop that we all had to attend?
What would you have done?
I was thinking about this scenario recently while reading about the latest standardized test cheating scandal. This one hit a little closer to home, since four of the “flagged” school districts are here in Washington. Apparently a district gets flagged if a significant number of classes have a suspicious spike in their test scores, a spike that can best be attributed to cheating. Getting flagged doesn’t prove there was a systematic posting of answers or teachers adjusting booklets after school, it just means that there was a weirdly suspicious uptick in test scores followed by a weirdly suspicious downtick the following year. And since real learning doesn’t usually follow that profile, the best explanation is that there was something fishy going on.
We’re hearing two fairly predictable responses. The pro-reformers want to turn the screws on the culprits while the anti-reformers are arguing that this is one more reason to abandon standardized, high-stakes testing.
Here’s what I think. First of all, we need to remember that any test is only a proxy for what it is we’re trying to measure. A standardized test isn’t the learning itself; it only represents that learning. And whenever we test something, we’re going to get one of four outcomes, illustrated in the following diagram:
TP stand for True Positive, an outcome in which the student really does have the knowledge or skills being measured and the test results bear that out. FP stands for False Positive: the student doesn’t have the skills or knowledge, but somehow the test results indicate that she does. FN is False Negative, where the student has the learning, but wasn’t able to show it on the test. TN, of course, is True Negative: The child didn’t learn it, and his test scores prove it.
Ideally, every kid in my room begins the year not knowing what I plan to teach them. They end the year knowing everything I taught. My job is to bring them from True Negative to True Positive. This is what I spend 98% of my time doing. In reality, however, some of my students learn the material for which they’ll be tested, yet because of the inaccuracies of the test, the stress of the situation or simply not knowing the academic language used in the questions, they move from TN to FN, an unfortunate outcome. Obviously I need to spend some time doing what I can to keep that from happening. This is called test-prep, and as long as I don’t go overboard, it’s a good idea.
The problem, however, lies in the FP quadrant. That’s where the cheaters lurk. If I spend 98% of my time teaching the essential skills and knowledge and 1.9% of my time teaching test-taking skills, I also need to spend 0.1% of time making sure that kids who don’t know the material aren’t cheating. And I need to make sure I’m not helping them cheat. We all do.
That’s why I like the fact that the Atlanta Journal Constitution is pursuing this issue. Up until now, there hasn’t been a good reason to keep cheating teachers and administrators from cheating. (Other than their own self-respect) Now we have one. If educators cheat, someone will notice. They’ll get “flagged,” or hopefully, fired.
We’re a nation of measurers. Testing is a fact of life. We need to accept it, embrace it, and do it correctly. And ethically. It shouldn’t take a newspaper article to keep rogue teachers and administrators from cheating, but so be it.
Oh, and Rachel? I didn’t help her. Maybe I should have, but I didn’t. I told her to read the question again and think it through, which is exactly what I was supposed to tell her.
And then after the test I told her what a “selection” is.
I would have told her, flat-out: selection means story. And I would have said it out loud, so other kids who could actually read the word “selection” would have a clue. But then, my idea of “cheating” is what we do to kids by forcing them to take tests when they’re 8 and–as is now policy or proposed policy in many states, making them repeat 3rd grade when they don’t pass.
Great blog, Tom.
I love that little grid.
I’m totally in favor of assessing kids, but I’m so frustrated with the tests we have and the way we use them. Washington has continued to change its tests so that it’s hard to prepare kids and it’s meaningless to compare one year to the next. All those years I prepared kids for extended responses on the reading assessment (the one that was called WASL but is now called HSPE)? Meaningless, because there are no more extended responses.
Another thing I have a serious problem with is that while we have “released” items for the state test, now the MSP or HSPE, we have no released items for the MAP. We’re told that if kids can read and do math, their skills will be measured by the MAP – we’re told that by the company, NWEA – but the questions on the MAP do not correlate with reading skills. My 7th graders were expected to know what a “malapropism” was in January. Do you know? I have yet to find a skilled reader adult who quickly remembers what that is. And I didn’t know they’d need to know that either, until I paced behind them in January, filling sheet after sheet of a legal pad with the stuff they were expected to know. Parallelism. Faulty logic. Red herring.
As if that weren’t enough, our assessments are scrambles – kids tossed around the building in a desperate attempt to find a computer that hasn’t crashed, or shuffled down the hall to the overflow room because mom’s car got a flat tire and the kid arrived late. And the test scores don’t really get the student anything. It would be great if the kids saw the tests as a way to measure their own progress, but my seventh grade student who just finished Wuthering Heights and was stumped by malapropism in January starts to figure out that whatever the test is measuring, it’s not her reading ability.
I have gone off on a rant. Your post is about cheating. I’m sure teachers are tempted, and I know some give in. There’s a lot of pressure on teachers in some buildings. It’s hard not to feel like a ridiculous word tossed inappropriately into the wrong age group is a cheap shot, and merits a cheap toeing of the line in revenge. But I don’t do it. At the end of the day, it’s not the test scores that matter. It’s that the child has learned. While I’ll continue to advocate for teachers to be willing to prove their students have learned, I’m not willing to advocate that our current standardized tests are an accurate measurement of that.
Mark, you’re damn right I did. There’s nothing subversive about it, either; it’s called “introducing academic language.”
I hope you enjoyed my selection.
Now here’s the rub: did you tell the next year’s third graders the meaning of the term “selection” before the test? I would have. I’ve done that for years. Each year, I get asked similar questions of the 10th graders I proctor. I answer as the rules tell me to. Then, the next year, during test prep I talk about all the questions kids had previous years. It feels like gaming the system, teaching to the test, whatever, but it feels a little subversive sometimes.