I like technology. Some say I adore it. But I do have limits. I am not bound by technology simply because something has a circuit board. For example, I have not found a decent calendar program that can display the many nuances that my monthly schedule takes. For this, I use paper. I make notes on paper, too. I gave up my PDA years ago after it failed to be as effective as paper. My PDA came close, but I knew it was not a match. I think knowing when to use a tool (and when not to) is crucial to efficiency.
I have my doubts on how far a computer can be used for assessment, but will keep full-fledge-hysteria at bay until the state finalizes its student assessment plan, reflecting Supt. Randy Dorn's desire to replace the WASL with a more efficient assessment tool. Part of this efficiency, it is said, will come from the use of computers. While the WASL had its drawbacks, mostly time and money, many have criticized the proposed WCAP as being a surface assessment, hardly worth being called an assessment.
I read an article last week that discussed how the WCAP could use a computer program to assess student writing. After the chills subsided, I reread the article. The more I hear about WCAP, the more questions I have, and rarely is that a good sign.
Can a computer program provide an accurate assessment of student writing? If the assessment is done on a computer, how many higher-level thinking questions will be removed because those questions require responses in written form? Do multiple choice questions show student reasoning, or just recall? Will subtle writing choices like figurative language, sentence structure, and word choice be registered with a program?
Time will tell.
In a state education system where the assessment is becoming more and more the final say, and one point can be the difference between "strategic" and "proficient", I am inclined to reserve my praise for the proposed system for assessing student writing with a computer program. Read the article and share your take?
@Bob, excellent list. I like your style (condensed, concise). And by-the-way, “Yes”.
Do I understand correctly? These are your suggestions for adjusting PC software for learning and teaching:
1. Make calendars more flexible to account for nuances of more teachers.
2. Increase software assessments of writing to include more features like authenticity, figurative language, sentence structure, and higher level thinking.
3. Increase accuracy of assessments.
4. Demonstrate more clearly the validity of computer assessments for future creative writing of test takers.
5. Clarify how individualization of learning and assessment works in a current classroom.
Please add to the list of what you think is missing from this summary. You never know who will read your blog post.
As we all know, whoever they are, they will want clear, concise, direct statements about what adjustments to consider when writing their software.
Yes?
Well…All of the concerns about the software aside. And concerns about having a dynamic classroom aside. What about the sheer number of students versus computers?
I’m not sure that Mr. Dorn has clearly thought that through, especially when you look at his quote. It would be nice if every one of our classrooms even had computers in them for a student to be testing in the back of the classroom while others were doing other work, but let’s face it there isn’t even that much room in many of the classrooms in my high school.
My husband is the Department Chair for our Business Department and is seriously concerned about what will happen when his 3 computer labs are called into service for this testing. That’s 90 out of the 150 computers we have on campus as labs. Where will his department teach? What about their curriculum? We have 600 freshmen this year.
Tom, I’m like you – I kind of like being able to tell the students that all of the schools and students in Washington are doing the same thing right now.
I like the idea on the surface, but the practice seems to be impractical.
Technology as an interface for facilitating assessment and feedback on student is fantastic; turnitin.com reviewing features and the comment features of MS Word as two very accessible examples. These are tools for teaching and helping students develop as writers because the use of technology allows meaningful feedback an formative assessment.
As for summative assessment, part of what makes writing effective or ineffective is the impact on the reader–a human–and I wonder if a program can be devised to identify a well-crafted, heart-tugging anecdote to support a persuasive point. Sure patterns can be recognized. Is the goal that students write in a manner which subscribes only to rigid patternology? I know we’re not looking for great literary essays from these kids, necessarily.
The human audience is what makes writing authentic and meaningful, even if the reader is assuming the perspective of a “pretend” audience (how many WASL readers were really considering moving to that student’s hometown those years ago…?)
It is not that computers cannot be used to assess student learning. They absolutely can: they can assess knowledge of content and to a certain degree critical thinking in choosing between options provided. However, reducing good writing to a series of patterns with X number of modifiers, semi-colons, and varied word choice would not be effective. One concept which I think would be nearly impossible for a computer to determine: verb tense shifts necessary within a text to illustrate past, present, conditional/subjunctive. How would the computer “know” what the student meant, and whether it was correctly applied? (a petty example, but an example nonetheless)
It is time in the discussion for me to say that depending on the use of the computer I could be for computer assessment of writing, time saver that it could be. Now here is my take, if the writing assessment (done by computer) were one piece of a more wholistic (collected works and assessment scores and data), then I would work with the group to make the computer assessment work. That’s one concession.
And as Bob pointed out, a computer program could be programed to recognize patterns. For the most part, those patterns could be found by the program and in the end have a high reporting rate. However, what about that small percentage of students whose writing is not recognized by the program; a small percentage could be a few thousand REAL kids. That part concerns me and this concern for the small percentage is why we have a court system that works the way it does so that, in theory, no one gets undue punishment.
A program could not possible account for every writing strategy or technique that a student employs, especially if it is timely or is the result of understanding some bit of information that is part of the current culture. A human reader would know that it is something and work with it.
The article I cited, states that adding a human reader increase accuracy. There is the percentage of students, real students, or the percentage of a student’s writing that is not understood. Put the human reader in for more accuracy. The article states that depending on the $$$$ the state could have the hunan reader or not. I am guessing that Washington, if it goes for computer based reading of writing, won’t “splurge” for the humans.
This is all good, but there is a greater, underlying concern of mine–that the act of writing, the reason to write, the process of writing will be lost in this system. A system that has a short writing assessment (writing has never been a slow process), a system that does not encourage the writing process (a state standard, but not enough time to do so in a short assessment), and a system that is read by a computer program (as a student I would be afraid to take writing risks in style for fear that it would not be understood by the program which is sad because writing really should be about risks in expression, finding that voice, making mistakes and deciding on a final piece).
So, if the computer program writing assessment team for Washington wants a skeptical member who loves writing, loves teaching, loves (adores) technology, shoot me an email. I will find time.
Bob, you sound chipper today. I enjoyed the optimism in your last comment. I share each and every one. Cheers all.
Thanks for posting another timely topic.
Yes, Teachers, new era schools exist in bits and pieces and more in some school programs than others. I like diversity!
Yes, Tom, some people imagine and commit uses of computers in schools differently from standard classroom teaching. That’s great!
Kudos, Teachers, for examining some of these new features. It’s great to see you addressing these changes.
Perhaps it’s time to say how you want computer automated testing and individualized student scheduling of taking those tests to work in your classroom?
And, yes, Travis, computers are designed to repeat patterns. If you want a certain pattern in English read and used by a computer, that can likely happen when you work with an education software developer.
You’re lucky to be so young and involved in education, a field likely to change more rapidly than even the most optimistic electronic communication equipment developers imagine!
It’ll be a great roller coaster ride for everyone, especially for learners, and will likely challenge policy makers’ standard procedures.
Yes?
I have not hear that throw away line. Scary.
Let’s take this scenario–some students can be doing their on-line assessment in the back of the room while the teacher teaches the rest of the class–one step further in thinking. I teach a dynamic class. It truly is English Alive! We are talking and engaging and if the teachable moment hits us, we go to the board as a class and work through some sentence structures. I would not be able to teach, in my usual manner, my high caliber manner, a manner that is for the benefit of the learning of the students, if I had students on computers taking an assessment while I taught.
I wonder if a computer would pick up on the thesaurus repetition for effect that I used in this last paragraph. I know my students could. And they would know why it is useful, much like they would know that it is not the preferred convention to not start a sentence with “And”, but it can be done so as a way of style when employed effectively.
I have a lot of concerns, too, Travis, including a major practical concern. One of Dorn’s throw-away lines is “some students can be doing their on-line assessment in the back of the room while the teacher teaches the rest of the class.” What!? One of the few things I like about administering the WASL is that every student in the state does it at roughly the same time. It makes it easier to get my kids to rise to the occasion. Which is nice, considering the fact that our school is judged by the press based on the results. With the stakes that high, I cannot imagine teaching a lesson with a third of my class trying to take a standardized test in the back of the room.
From facebook: That’s sort of like picking a house plan out of a plans book in the architecture world. 🙂