STEEP THROAT EXPLAINS WHERE GAINS COME FROM! Uh-oh! A testing executive shared his views about where steep score gains come from: // link // print // previous // next //
THURSDAY, JULY 7, 2005
MASTERWORKS OF SELF-INTEREST: Astonishing. The Washington press corps always astounds us when its interests or preferences come into question. But even by the corps own standards, todays editorials and news reports about Judith Miller are masterworks of self-interest. For example, try to believe that the New York Times included this in todays editorial:
NEW YORK TIMES EDITORIAL (7/7/05): Mr. Fitzgerald's attempts to interfere with the rights of a free press while refusing to disclose his reasons for doing so, when he can't even say whether a crime has been committed, have exhibited neither reverence nor cautious circumspection. It would compound the tragedy if his actions emboldened more prosecutors to trample on a free press.Fitzgerald can't even say whether a crime has been committed? Duh. Thats what hes trying to determine! And has Fitzgerald refus[ed] to disclose his reasons for interfering with poor trampled Miller? Earlier in the editorial, the Times seemed to explain this weird statement:
NEW YORK TIMES EDITORIAL: Ms. Miller did not write an article about Ms. Plame, but the prosecutor, Patrick Fitzgerald, wants to know whether anyone in government told her about Mr. Wilson's wife and her secret job. The inquiry has been conducted with such secrecy that it is hard to know exactly what Mr. Fitzgerald thinks Ms. Miller can tell him, or what argument he offered to convince the court that his need to hear her testimony outweighs the First Amendment.Oh! Its hard to know exactly what Fitzgerald thinks! Indeed, its hard to know because his inquiry has been conducted with secrecy—because hes actually been doing his job! But right there in their own editorial, the Times explains why Fitzgerald is interfering with Miller—he wants to know whether anyone in government told her about Mr. Wilson's wife and her secret job. Everyone knows this, of course, but so what? Three paragraphs later, the Times pretends it doesnt know what it explains in this earlier passage. But so it goes as a perfumed elite endlessly pimps its self-interest.
Where does wisdom lie in this complex matter? That is a matter of judgment. But you wont get any complexity here! In a word, the Times editorial is embarrassing—a world-class example of pure propaganda. If you want to know why some average people think this press is a powdered elite, just read through this awful work and imagine how its self-dealing logic must seem to these average citizens.
But then, this piece hardly stands alone; the Post writes a piece that is equally foolish, but is mercifully shorter. Heres one example of the rags deathless logic:
WASHINGTON POST EDITORIAL (7/7/05): Mr. Fitzgerald is charged with investigating whether the leak of Ms. Plame's identity by one or more administration officials to columnist Robert D. Novak two years ago constituted a criminal act. Yet so far the only person to actually face jail time in his prolonged investigation is a reporter who never wrote a story about Ms. Plame, let alone revealed her CIA affiliation. While an investigation of the leak was justified, it is far from clear—at least on the public record—that a crime took place.Pathetic, isnt it? [I]t is far from clear—at least on the public record—that a crime took place! But Fitzgerald isnt working from the public record; the Post doesnt know the state of the private record, but is eager to peddle this pap all the same. And good God! The paper pimps the dumbest misdirection of all, noting—for the ten millionth time—that Miller never wrote a story about Ms. Plame. As even this papers daft editors know, this accurate fact is completely irrelevant to the actual questions at hand. But so what? The press corps self-dealers just wont stop reciting it. Cant you just hear them? Hey, rubes!
If youve watched this case discussed on cable, youve likely seen a stream of one-sided discussions, in which the corps perceived interests are pimped. Yesterday afternoon, we were stunned by the discussions on MSNBC; Stalin himself must have cheered the way all balance was airbrushed away, and our analysts groaned and covered their eyes when Norah ODonnell attacked Fitzgerald for the rabid way he has acted. But then, weve routinely seen it, for years: When the press corps interests and preferences are at stake, they will issue a blizzard of spin. If you want to know why average people see this gang as a perfumed elite, read through the ludicrous, one-sided work theyre churning about this complex matter.
TOMORROW: Why didnt you hear about all those waivers? Stuart Taylor answered that question—in April 2004!
PART 3—STEEP THROAT EXPLAINS WHERE GAINS COME FROM: Were New York Citys steep score gains real? If the lives of urban kids really matter, this question is very much worth exploring. As weve seen, the citys fourth graders did much better in this years high-stakes ELA exam (English Language Arts). Indeed, 59.5 percent achieved proficiency; this was a jump of 9.9 points from 2004, and it was the highest passing rate in the tests seven-year history. But uh-oh! The passing rate jumped by 8.2 points across the state of New York as a whole, and systems in Yonkers, Rochester and Syracuse had shown more gains than New York City. But when a city councilwoman dared suggest that a panel should study the citys test scores, Lori Mei, director of testing, had a familiar reaction (see THE DAILY HOWLER, 7/6/05). Michael Winerip reported the action:
WINERIP (6/29/05): When Ms. Moskowitz countered that the city tests are hard to monitor since they're not made public, Dr. Mei deferred to representatives from Harcourt Assessment (which does the city English tests) and CTB/McGraw Hill (math), who testified that the city tests were scientifically scaled.It was an age-old reaction: Trust us, the bureaucrat said. Everyone—even the testing companies!—say the tests are well-constructed, Mei said. Everyone—even the testing companies!—say that the score gains are real.
But theres one big problem with Meis response. All her authorities stand to gain if those pleasing test scores are real. The mayor has said that the score gains are real—but then, hes running for re-election. The test companies say that the score gains are real—but they are the ones who were paid to devise them. And Dr. Mei says that the score gains are real—but she gets fired if theyre phony. Trust us, the good doctor said. Trust us—even though everyone I cite is an interested party.
Yes, an array of powerful interests gain when city test scores show a steep rise—even if the rise is phony. Mayors gain. Superintendents gain. And teachers get to ask for more money! The only ones who wouldnt gain from a hoax are the urban children themselves—the kids whose lives have often been used as the platform for the fraudulent test scores. Even in the case of a hoax, many people gain from good test scores. Which brings us to something we were told long ago about where some score gains may come from.
On a personal level, we had known about hoaxes in the Baltimore schools since the 1970-71 school year. (We had several friends in a widely-praised school—a school which was blatantly cheating, in every way possible.) We had written on this general problem for the Baltimore Sun and Evening Sun. We had consulted with Dr. John Cannell, who created a nationwide flap with his report, The Lake Wobegon Effect (see THE DAILY HOWLER, 4/19/01.) And in the course of our research, we had developed a telephone friendship with a very high source—a very high-ranking executive for one of the nations most widely-used standardized test programs. And uh-oh! On one occasion near the end of our relationship, this high-placed fellow—lets call him Steep Throat—gave us an unforgettable lesson in where some steep score gains may come from.
On this occasion, Throat surprised us by revealing that he would soon be stepping down from his post. The reason? He felt that his company could no longer compete with a rival test battery. Over the course of the previous several years, the rival battery had been taking market share from Steeps well-known standardized test. One major reason? As everyone knew, systems which used the rival test tended to crank out some very good test scores! On this day, Steep Throat told us something that people suspected within his own company; they suspected that the rival company was faking its norms in order to generate those pleasing, high test scores. We cant compete unless we fake our norms too, Steep said. And so he was giving up—changing careers.
(A minor technical point: In a norm-referenced standardized test—the type of test then at issue—a students score is compared to a nationwide norm group—a group of children who took the test while it was being constructed. A students score only has meaning when its compared to the performance of this national group. But uh-oh! If a test company wants to do so, it can publish massaged, phony norms, thereby creating a test which produces artificially high scores. Steep Throat told us that his company now suspected this was occurring. Theres no other way the rival test could produce such good test scores, he said.)
Was Steep Throat right in his suspicions? We have no way of knowing—not a clue. But at the time, the rival test was gobbling up market share, especially in low-scoring urban systems. Many city systems were switching to this test—and everyone knew that this was happening because the test produced high scores. Was Steep Throat right in his suspicion; were these pleasing scores phony? Again, we have no way to tell. But even twenty-five years ago, a very bright man at the top of one company believed that such faking was underway—and that it was changing the face of American testing. And one other thing was perfectly clear—the rival test company was making big profits by virtue of the pleasing scores it produced. Major money was changing hands because this test produced high scores. What happened when you switched to this test? Mayors got to hold pleasing press conferences—and the rival testing company raked in some big dough.
So this is why our analysts laughed when they read Dr. Meis absurd comments. Trust us, the mayors aid said. Trust us—the people who gain when those test scores go up. The mayor gets re-elected—trust him! I get to keep my job—trust me! And oh yes! The testing companies get pleasing new contracts! Trust them, the mayors aid said.
Readers, Tobias had a very good idea when he suggested that it would be useful for the city to appoint an independent panel to analyze test results. But a range of powerful interests gain when those test scores go shooting up—and the lives of urban kids tend to come last when these parties smell their self-interest. What happened when New Yorks test scores went up? Bloomberg got to take deep bows—and the companies got to sign new contracts. But sadly, when Moskowitz and Tobias raised excellent questions, the New York Times pooh-poohed their concerns (see THE DAILY HOWLER, 7/6/05), just as big papers have tended to do over the course of the past several decades. Do the lives of urban kids matter? More on this question tomorrow—but over the course of the past forty years, the answer has been rather clear.