By Jim Schutze
By Rachel Watts
By Lauren Drewes Daniels
By Anna Merlan
By Lee Escobedo
Defenders of TAAS and partisans of Gov. George W. Bush have claimed the Rand study was flawed because TAAS measures only knowledge of the Texas state public school curriculum, while the other tests considered by Rand look at what children are taught nationwide.
In any event, the TLI is not George W. Bush's invention. Mandated by law, it is carried out according to a practice called "equating," which is widely used in the testing industry and generally accepted as legitimate. An oversimplified explanation would be that questions for each year's new test are field-tested to see whether they are harder or easier than the questions on the 1994 test. The purpose is to adjust the toughness or easiness of each year's test back to an original standard, so that kids don't look dumber one year just because the questions on their TAAS test happen to be harder than the ones that kids were asked on the same test last year.
From the time these adjustments were first ordered by the Legislature in 1994 until 1999, the equating studies found that no adjustment at all was needed on the vast majority of tests in most years. In a few instances, only very minor adjustments were made, most of them slightly upward, so that it took a little bit better than a 70 percent raw score to get a TLI score of 70 percent.
The anomaly in the data sent to Garcia has to do with what happened in the fall of 1999, when raw scores on all of the state's TAAS tests suddenly plummeted. The equating studies were finding that scores needed to be cranked back up 10 to 20 percentage points in order to guarantee a uniform level of difficulty back to 1994, considered "Year One" of the current accountability cycle.
TEA spokeswoman Ratcliffe says TAAS tests in 1999 became "more rigorous" because of a legislatively mandated change in the basis of the test. Questions on earlier TAAS tests had been keyed to something called "Essential Elements" (EE)--a list of things teachers were required by law to present to their students in each course. By 1999, the state had switched over to a new list of things to be taught, called the "Texas Essential Knowledge and Skills (TEKS)." The new list is supposed to be more rigorous, Ratcliffe says, and therefore the test based on the new list is tougher.
But it is at this seam that the political element intrudes on the landscape of TAAS. When Texas, in its tough-minded desire to force schools to teach more, adopted a more rigorous required curriculum and then keyed statewide tests to it, why shouldn't Texas have expected fewer kids to pass those tests? At least in the first few years, wasn't it reasonable to expect kids to get lower scores?
In fact, raw scores did fall dramatically after the switch to TEKS. But in a letter to the state's public school administrators a year ago, Texas Education Commissioner Jim Nelson promised them that the TLI equating studies would erase the effects of the tougher new curricula and tests.
"Since a child who could have passed last year's test will also pass this year's," Nelson assured the troops, "there will be no change from the perspective of a school district for purposes of accountability."
That was cheery news, no doubt, for the state's public-school administrators, who can win $25,000 bonuses in some districts for good pass rates on the TAAS and can suffer serious negative effects from low pass rates.
Not everyone is cheered by the TLI. Walter Haney, a Boston College education professor who spent two years analyzing the TAAS, is skeptical about the use of equating to bridge two tests after the underlying basis of the tests was changed from EE to TEKS.
"There's something extremely fishy going on with regard to the equating on the TAAS," Haney says. "Theoretically, you can't equate two tests unless the content specifications are the same."
But every attempt to quibble on the equating process seems to beg an overarching question: In a transitional period when the curriculum and the tests have been made harder, presumably because Texas is demanding more of its students, why should the raw scores be heavily manipulated to produce the appearance of a flat level of difficulty?
TEA test expert Keith Cruse says, "That's strictly a policy question. You would have to talk to the State Board of Education about that."
Texas Education Commissioner Nelson says that there will come a time fairly soon when the TEA will suspend its efforts to equate every test back to the 1994 benchmark of difficulty and will start all over from a new "Year One."
"Come 2003, we're going to finish this cycle of our accountability system and move, because of the requirements of Senate Bill 103, to a whole new system. We will have a new Year One."
Major movement in the system of statewide testing in Texas is directed by law and does not happen at the discretion of Nelson or the TEA. It certainly can't be fiddled at the last moment to help a political campaign, even a presidential one. But the governor and the Legislature did have an option, in designing this system, to suspend the equating process and begin a whole new cycle last year when the basis of the test was changed.