By Jim Schutze
By Rachel Watts
By Lauren Drewes Daniels
By Anna Merlan
By Lee Escobedo
By Eric Nicholson
While our governor has been out running for president on the strength of a dramatic rise in test scores for Texas schoolchildren, it would have been helpful to know that students who took the last statewide math test received a passing score of 70 percent for answering 50 percent of the questions correctly.
This represents a major and recent change. In the fall of 1998, a student had to answer 70 percent of the questions on the Texas Assessment of Academic Skills correctly to get a score of 70 percent.
The fact that any adjusting of scores is going on at all--especially of the magnitude occurring in just the last year--is troubling to some test-watchers because the Texas tests originally were billed as straight "criterion-referenced" tests. A criterion-referenced exam is like the quiz a geometry teacher writes: That is, the teacher draws up a list of 10 things students should know and then asks 10 questions to test them on it.
That's supposed to be very different from a "norm-referenced" test, where the point is not to add up what students know but to see where students stand in relation to other children their age. What seems to have changed, with very little public fanfare, is that the no-nonsense criterion-referenced TAAS is being heavily normed.
When TAAS was initiated, the results had a more commonsense ring. It was a true criterion-referenced test, and scores originally were reported as so-called "raw" or real scores: The percentage of answers correct was the score. In the flinty rhetoric of educational "accountability" in Texas, the TAAS test was a "what-you-see-is-what-you-get" deal. But in 1994, in response to fretting from school administrators and teacher groups that felt some TAAS tests might be harder year-to-year than others, the Texas Education Agency began applying the Texas Learning Index (TLI), an arcane formula designed to smooth out scores in case one year's new test was slightly easier or harder than the previous year's.
Even at that, the annual TLI adjustments were minor--barely fractional to non-existent--until the fall of 1999. Suddenly the adjustments became major, raising the question in some observers' minds of just what the manipulations are really designed to do. Are they supposed to make one year's test exactly as hard as the last year's would have been for the same student? Or are they designed to crank out a certain number of passing scores no matter what the children know?
Some respected authorities in the field of testing say the so-called "Texas miracle"--a general improvement in scores since testing began--is absolutely not a hoax, and the adjustments that the state has made since 1994 are legitimate, even necessary calibrations to make the test a more accurate measurement of student achievement. But other authorities with rank and prestige in the field say they think the adjustments are suspect, especially given the timing.
Unfortunately, there has not been time for a healthy debate on the issue of TAAS score adjustments because it was only in the last few weeks that the precise scope of the adjustments became known. On October 20, state Rep. Domingo Garcia, a longtime critic and watchdog over the Texas Education Agency, received a set of numbers from TEA showing the adjustments. Garcia, a Dallas Democrat who has led the charge against TEA on several issues related to minority students, received data on the TAAS score adjustments in response to a formal demand under the Texas Public Information Act.
Debbie Graves Ratcliffe, spokeswoman for TEA, says Garcia did not have to file a formal request and could simply have asked for it as a legislator. "We would have given it to him," she says.
But Garcia says his staff gave up on normal channels after a year of efforts to get specific answers to specific questions, especially whether annual adjustments were being made in the percentage of correct answers required to win a passing grade of "70 percent" on the TAAS. The information is coming out now, he says, because it took him this long to get it out of TEA.
The existence of a formula for adjusting TAAS score has been no secret among education experts in Texas, but it is certainly little-known to the public.
Garcia does not claim to see a smoking gun in the information he has received. What he thinks he does see is smoke.
"It looks funny that a 50 percent is a 70 percent," he says. "It looks funny that the time when they started doing these major changes is a year ago when the governor started running for president.
"But my questions are not really about that. My questions are whether the kids are really learning, as we are told, and whether they are really being prepared to get into college and to do well in college. Or are we artificially inflating scores for political purposes at the expense of our children?"
Garcia says the urgency of these questions is sharpened by the recent Rand Corp. report that found the achievement gains attributed to TAAS scores in Texas not only are not supported by other tests but are in some key instances contradicted. Rand researchers found some national tests show much slower achievement growth in Texas, and even some Texas tests show a decline in overall college preparedness of Texas public high-school graduates.