By Stephen Young
By Stephen Young
By Stephen Young
By Jim Schutze
By Rachel Watts
By Lauren Drewes Daniels
So guess what happens? After DISD uses the "hard," or real, results to black-mark the teachers, it makes a biiiig adjustment in the grades. On a pre-advanced placement chemistry exam, a grade of 40 percent on the ACP is adjusted to 70 percent when it is reported to students and parents. A score of 60 percent--10 points below the real pass mark--becomes an 80 on a student's record. Your kid got a B, the smart little devil!
Instead of having to tell the pubic that all but one school flunked the Algebra I exam, DISD juices up the grades and reports that 50 percent of the schools made passing grades and the rest of them were all crowded up there to within a few percentage points of passing.
The hocus-pocus actually gets worse here. There are two kinds of tests out there. One is called "criterion-referenced," which just means it asks a bunch of questions to see if you know stuff, and you get a straight grade based on how much of it you know. The other is called "normed." A normed test is a version of the old grading-on-the-curve deal. Your grade doesn't reflect how much of the stuff you actually know; it reflects how well you did in comparison to everyone else.
I'm told that those two kinds of tests are put together very differently. They are different critters. But the DISD tries to work the ACP both ways. They build it as a "criterion-referenced" test, use it that way to put the black marks on the teacher files, and then they pump the grades full of air for public consumption. When it goes public, they say the test is "normed."
When I showed this memo to a teacher friend of mine, he became extremely angry. He said he couldn't talk about this with me on the record, because he feared reprisals against his students (this is not a paranoid guy).
Speaking on background, he did say: "Under that system, nobody's accountable except teachers and students. We're responsible. But the people who put us out there in the portable with the kids and the list of 129 different course objectives that we're supposed to teach them in 177 instructional days, they're not accountable."
I sat down with Robert Mendro, a nationally respected testing and education statistician in DISD's division of evaluation, accountability, and information services. This was his memo. He started by giving me some of the standard lines about how the test is criterion-referenced but then it's also sort of normed later on.
I asked him this: If you know that a kid has answered only 40 percent of the questions on the pre-AP chemistry test correctly, why would you tell him and his parents that he passed the test?
Mendro looked at me for a long moment. He said, "I don't suppose it sends a very good message, does it?"
Over the course of last week, I discussed this memo with a number of teachers here in Dallas and with some testing experts around the country, several of whom wanted to make an additional point--something they felt ran deeper than the mere skulduggery exposed in the memo.
Alfie Kohn, a well-known author and critic of standardized testing who lives in Belmont, Massachusetts, e-mailed me: "I'd hope you can help your readers keep their eyes on the big picture: The issue isn't just this dubious scoring practice but what it reveals about the inherent arbitrariness--the utter lack of objectivity--about standardized testing and the way it fails to provide meaningful accountability (while, in the process, disrupting and distorting classroom instruction)."
Before the statistical dust storm starts--which I fully expect as soon as this column appears--let's take a big reality check here. The state administers an Algebra I exam that is separate from the TAAS tests. It's a basic "criterion-referenced" do-you-got-it or do-you-not-got-it test. Dallas students pass the statewide algebra test at half the rate for students in the rest of Texas. Only one in four Dallas students can pass the test, according to the last round of scores.
But all year long before they take the statewide test, Dallas students tend to bring home passing grades in Algebra, according to a report provided me by DISD.
Robert Mendro didn't invent this policy. His office generates the equations that "scale" the ACP scores. This was his memo. But he and the people of his division are only following orders.
I don't know whose orders. I don't know where this started or when. But this stinks. This hurts children. We should all be ashamed.