Joel Klein gave a striking reply: Two Saturdays ago, we reviewed Dianne Ravitch’s op-ed piece, in the New York Times, about the New York City schools. See THE DAILY HOWLER, 4/11/09. For Ravitch’s piece, just click here.

On April 16, the Times published a letter from Joel Klein in response to Ravitch’s piece. Klein is chancellor of New York City Schools. We thought one part of his response was especially striking—was truly remarkable.

Quick overview: About half Klein’s letter dealt with the topic of drop-out/graduation rates. As we noted, this is a difficult technical matter. We’d like to see the New York Times do more reporting on it.

The most striking part of Klein’s response came in the passage we offer below. In the first paragraph, he comments on New York City’s scores on the National Assessment of Educational Progress (the NAEP), the so-called “nation’s report card.” In the second paragraph, he comments on New York City’s recent performance on the state of New York’s statewide tests:

KLEIN (4/16/09): The national tests she cites [the NAEP] are not the measure of federal accountability, are given only to a small sample of schools, and are not aligned with New York State standards and therefore with what we teach in our classrooms. (That said, our fourth-grade scores on those tests are strong.)

New York City's gains on state tests have substantially and consistently exceeded gains made throughout the rest of the state during mayoral control. Even if those tests have gotten easier, as Ms. Ravitch claims without evidence, they have gotten easier for everyone.

Some of Klein’s comments about the NAEP strike us as slightly odd. That said, we did think that Ravitch’s piece tended to understate New York City’s success on those tests. “[T]he statewide scores on the N.A.E.P. are as flat as New York City's,” Ravitch wrote in her piece. For ourselves, we’d regard that statement as a bit of a stretch. Having suggested the NAEP is no doggone good, Klein obliquely challenged Klein’s gloomy assessment.

But the statement by Klein which we have highlighted struck us as truly remarkable. In that paragraph, he is discussing the annual statewide tests administered by the state of the New York. In some recent years, questions have surfaced about those tests. Scores have risen across the state, with some teachers claiming that scores had risen because the tests had gotten easier. (To recall one example, see THE DAILY HOWLER, 10/06/05). Because New York City has played a key role in recent debates about testing and standards—because sweeping claims have been made about what New York City’s scores gains mean—these are very important questions.

That’s why we were struck by the statement we have highlighted.

Klein claims that New York City’s recent score gains have been “substantially” larger than those of the state as a whole. Because the state of New York maintains a gruesome reporting system, we have given up on trying to fact-check this claim. Other states—Maryland, North Carolina, Washington—maintain highly informative, user-friendly web sites to report statewide test scores. The state of New York is a whole different ball game. This particular state almost seems determined to keep its results in the dark.

That said, note Klein’s comment about the claim that New York’s statewide tests have gotten easier.“Even if those tests have gotten easier, as Ms. Ravitch claims without evidence, they have gotten easier for everyone,” Klein wrote in his letter. (“Our state tests are, unfortunately, exemplars of grade inflation,” Ravitch had written.) We don’t mean this as a criticism of Klein, but that is a truly remarkable statement. Even Klein, the chancellor of New York City’s schools, is unwilling to say whether New York’s statewide tests have gotten easier!

Klein is a very key player—and he’s very smart. Even he doesn’t seem to know if the state’s tests have been “dumbed down!”

Let’s repeat a few key points about this essential question:

This question, which is very basic, shouldn’t be a matter of guesswork. Any test-maker should be able to demonstrate that a given year’s tests are as difficult as the previous year’s tests (are “equivalent” to the previous year’s tests). This should be a statistical question, not a matter of guesswork or judgment. (In a technically competent world, test items themselves should get tested.) When even Klein can’t speak to this question, it means there is a major problem in the state of New York. (Just a guess: A similar situation obtains in other states.)

Let’s state the blindingly obvious: Year-to-year comparisons make no sense unless we know that this year’s tests are as difficult as those from last year. And yet, journalists endlessly make such comparisons without making any attempt to address this basic question. In his letter, even Klein wasn’t willing to say whether these tests have gotten easier. In that moment, you see the Potemkin state of much of our current discussion.

Alas! We live inside a set of “Truman shows”—an endless set of feigned discussions. Klein doesn’t know if these tests have been dumbed down. Well guess what, folks? You don’t know either!