REFORM FIRST, EVIDENCE LATER! Miller is fervent about a reform. But would his hot new approach work? // link // print // previous // next //

A fascinating, maddening report: The good news: The New York Times is doing a lot of work these days about public school issues.

The bad news: It can be maddening to read their reports, given the frequent technical blundering. A few weeks ago, Sam Dillon made a cosmic blunder in a lengthy front-page report (see THE DAILY HOWLER, 10/8/10). Today, we occasionally gnashed our teeth as we read this report by Sharon Otterman.

Otterman reports on Geoffrey Canada’s ballyhooed “Promise Academy” schools, part of his “Harlem Children’s Zone.” Canada is one of the stars of Davis Guggenheim’s Waiting for Superman; in NBC’s recent teacher-bashing special broadcasts, Canada bashed teachers with the best of them. But how well have his schools been doing? Otterman attempted to say, but in various ways, she failed.

In some ways, Otterman seems to suggest that Canada’s schools haven’t done all that well. Here you see her first attempt to quantify performance. She is discussing Promise Academy I, one of Canada’s two Harlem schools:

OTTERMAN (10/13/10): On a recent Thursday, the current high school students, neatly attired in blue and white uniforms, got special help in college note-taking skills, and chatted animatedly about velocity in an advanced physics class….

“You really have to put money into personnel,” said Marquitta Speller, who has been the high school principal since January. “I don’t think you can experience the same level of success without the same level of resources.”

But most of the seventh graders, now starting their third year in the school, are still struggling. Just 15 percent passed the 2010 state English test, a number that Mr. Canada said was “unacceptably low” but not out of line with the school’s experience in lifting student performance over time. Several teachers have been fired as a result of the low scores, and others were reassigned, he said.

Fifteen percent passed the state reading test? That sounds very low. (Presumably, Otterman is discussing the performance by those seventh graders on their sixth grade tests, back in April.) But 2010 was the year when New York State suddenly made its statewide tests “harder;” Otterman never says what the passing rate was in New York City or New York State as a whole. How did that group of students do as compared to other sixth graders? There’s no way to know from this report. Absent that info, the passing rate doesn’t really tell you much about the relative success of this school.

As she continues, Otterman does somewhat better. In this passage, she provides the basis for making some comparisons, withholds the basis for others:

OTTERMAN (continuing directly): Giving administrators the ability to fire teachers for poor performance is one of the central suggestions of “Waiting for ‘Superman.’ ” Overall, 38 percent of Promise Academy I’s students in third through sixth grade passed the 2010 English test under the state’s new guidelines, placing it in the lower half of charter schools citywide, and below the city’s overall passing rate of 42 percent. In Harlem as a whole, just 29 percent of children passed.

Promise Academy II, an elementary school that occupies part of a public school building, did better, with 62 percent passing in English, among the top 10 percent of charters…Both schools continued to outperform the city in math, with 60 percent passing in one school and 81 percent in the other.

In the highlighted passage, Otterman provides the types of data which let readers make relevant comparisons. In grades 3-6, on the statewide reading tests, Promise Academy I did somewhat worse than New York City as a whole, somewhat better than the rest of Harlem. (She didn’t provide the statewide passing rate.) Promise Academy II outperformed the city, by a substantial margin. But by the second paragraph, we are once again deprived of a basis for making real comparisons. Both schools “outperformed the city in math,” we’re told. But by how much? By two points? By a lot? We aren’t allowed to know.

Otterman covers important ground, something the Times has been doing lately. But the technical bungling is never far off in this paper’s educational reporting. By the way, how does Otterman know that the highlighted claim is accurate?

OTTERMAN: [Promise Academy I], which opened in 2004 in a gleaming new building on 125th Street, should have had a senior class by now, but the batch of students that started then, as sixth graders, was dismissed by the board en masse before reaching the ninth grade after it judged the students’ performance too weak to found a high school on. Mr. Canada called the dismissal “a tragedy.”

On a recent Thursday, the current high school students, neatly attired in blue and white uniforms, got special help in college note-taking skills, and chatted animatedly about velocity in an advanced physics class. Most were well below grade level when they first got to the school and took three or four years to catch up; many are now ahead.

Did this school really dump a whole year’s worth of eighth graders because they weren’t doing well enough? If true, that’s a remarkable story. But how does Otterman know that the highlighted claim in that passage is accurate? That’s the kind of claim schools like this love to make. But how do we know that it’s accurate?

Did Otterman trust but verify—check this claim against actual data? Or is this just something she was told? Even at the top of the heap, education reporters still don’t feel the need to let readers know.

Special report: Matt Miller’s strange tale!

PART 2—REFORM FIRST, EVIDENCE LATER (permalink): Matt Miller probably isn’t from Mars, though he did a fairly good impression in Sunday’s Washington Post. His piece topped the paper’s “Outlook” section, positioned next to a “manifesto” about public schools from a familiar, shopworn gang of educational reformers.

Those fiery authors included Joel Klein, chancellor of New York City’s schools. In a rational world, Klein would be announcing his resignation in the wake of Jennifer Medina’s long, embarrassing news report in Monday’s New York Times (see THE DAILY HOWLER, 10/12/10).

Compared to Klein and his sovereign (Mayor Bloomberg), Miller comes off as almost human, though we did have reservations as we read his piece. Our reservations leaped to life in his closing paragraph.

In his Outlook piece, Miller pimped an upscale version of the latest hot educational fad. He very much thinks the public schools would benefit from better teachers. Well—not from better teachers, exactly. Miller thinks the public schools would benefit from smarter teachers—teachers drawn from the upper third of their college class.

On the surface, this seems to make perfect sense. Why wouldn’t it be a good idea to attract “smarter” teachers?

On Sunday, Miller and his co-author, Paul Kihn, oohed and aahed about this proposal, driving the latest very-hot fad as our “intellectual leaders” pretend to address the challenges facing our schools.

Miller and Kihn went on and on, discussing how great it would be if we could recruit smarter teachers. They dreamed of various strategies which might lure top students into the classroom; their piece sat next to Klein’s “manifesto,” which trashed current teachers well. (And their infernal unions. More on Klein’s piece tomorrow.) But because we’ve actually worked in low-income public schools, a question kept popping into our heads as we read Miller’s piece.

Yes, it surely seems to make sense; it seems to make sense to lure “smarter” people into the teaching profession. But what makes Miller so darn sure that this strategy would actually “work” in some significant fashion? That it would work in low-income elementary schools, where we face our greatest challenges? What makes him want to promote the idea that this approach is the “silver bullet?” What draws him to this teacher-based approach, in lieu of everything else?

Miller cajoled and implored throughout, sounding very human. But because we’ve actually worked in low-income schools, we had our reservations. We also had some reservation because we’ve looked at some actual studies. And sure enough! Finally, at the start of his very last paragraph, Miller coughed this up:

MILLER (10/10/10): Some U.S. researchers say there's little evidence that teachers with stronger academic backgrounds produce higher student achievement…

That came right after Miller said that “the social and economic returns could be enormous” if we recruit smarter teachers (our emphasis).

Miller is certainly right on one score. The social and economic returns could be enormous—but then again, the returns could be minuscule! And good lord! If “some researchers” say there’s little evidence, shouldn’t somebody check their claims before we go stampeding off? In her recent column on Waiting for Superman, Gail Collins did make one excellent point. She offered this account of the way these things have frequently worked:

COLLINS (9/30/10): Older teachers tend to respond to calls for education reform with cynicism because they've been down this road so many times before. In 1955, a best seller, ''Why Johnny Can't Read,'' stunned the country with its description of a 12-year-old who suffered from being ''exposed to an ordinary American school.'' Since then, the calls for reform have come as regularly as the locusts. Social promotion has been eliminated repeatedly, schools have been made bigger, then smaller.

It’s true. Quite routinely, someone comes up with some new silver bullet; we stampede off in that direction, with our “experts” running hard to get to the front of the queue. Today, the “solutions” all seem to revolve around the idea that our teachers, and their infernal unions, have somehow caused the massive problems found in our low-income schools. (Unmistakably, this simplistic message has been folded into a decades-old war directed at unions in general. In making this complaint, the reformers never mention the actual rise in test scores these infernal teachers have helped produce over the past dozen years.) Miller offers a courteous, upper-end form of this current silver bullet. But like the other faddish reformers, he discusses teacher improvement and nothing else, giving the unmistakable sense that this will solve our problems.

Will it? In some situations, “smarter” teachers would probably help; being “smart” is an advantage in all sorts of circumstances. But is academic ranking really the answer to the challenges we face in our low-income elementary schools? As we read Miller’s screed, we kept thinking of those studies of Teach for America—studies which seemed to show that the bright young Princeton kids didn’t do better than regular teachers, except perhaps at the high school level, pretty much as you might expect.

Those studies seemed to suggest that academic smarts may not be a silver bullet. But so what? The Bloomberg/Klein/Charlie Rose crowd is in love with the idea of the bright young Ivy Leaguers condescending to save the black kids. (For certain elites, this has been a pleasing novel for at least forty years.) They have pimped TFA to the ends of the earth, even when studies don’t support the wildly embellished claims the org makes on its own behalf.

TFA tends to embellish a lot. This seems to be fine with the swells.

We’ll admit it—we recalled those studies as we read Miller’s true-believing piece. And then, hot damn! Right there in his closing paragraph, he finally gave us the semi-bad news—and of course, like all space invaders, he promptly flew to Finland! Here is Miller’s full closing paragraph, in which he returns to a theme he pushed all through his piece. Finland and Singapore do it, he says. That makes it the answer for us:

MILLER: Some U.S. researchers say there's little evidence that teachers with stronger academic backgrounds produce higher student achievement, but this conclusion is starkly at odds with the experiences of Singapore, Finland and South Korea. Our McKinsey colleagues have studied more than 50 school systems around the world and have never seen a nation achieve or sustain world-class educational performance without drawing its teachers from the top third of their class. Should we really bet our children's future on the possibility that our country might be the exception?

Correlation isn’t causation, of course. Except when experts and swells—and space invaders?—get on their latest roll.

All through his piece, as it has been scripted, Miller describes the wonders of Finland. If Miller is a space invader, could this be the tip-off? Could this be the flaw in the programming of these human-seeming visitors? By some minor technical error, have they perhaps been programmed to discuss things in Finland too much?

Back to Miller’s closing paragraph, which is hugely important:

McKinsey is a think tank; it has prepared a report on this topic. Finland is a middle-class, high-literacy nation which has done good things with its schools. For ourselves, we wouldn’t oppose an attempt to lure “smarter” students into the classroom—but will that strategy produce good results in our nation’s low-income schools? We don’t have the slightest idea, although our experience suggests that there are other, very major factors that teachers can’t address by themselves, no matter how good their college grades were. And we’re sorry, but you can’t find out by flying to Finland. Or even to Singapore.

Here at THE HOWLER, we would support Miller’s basic idea. Presumably, it’s a good idea to lure smarter college students into the teaching profession. But a camp-meeting fervor surrounds the current, rolling attack on our teachers—and, as noted, it’s neatly packaged with a decades-long war against unions in general.

It might be better to get better teachers—but would that approach be enough? In the case of Miller’s specific idea, would it work in our low-income schools? In our low-income, first-grade classrooms? And by the way: Are there any other approaches which might also be pursued? In the current teacher-trashing environment, gangs of roving superintendents, “reformers” and “experts” blame our problems on bad teachers. As you’ll note, they rarely suggest that we look for ways to attract smarter people to the jobs they hold.

Miller wants to attract smarter teachers. But when it comes to our struggling low-income schools, does he actually know what he’s talking about? Or is he just chasing the current fad? With single-minded devotion, he pursues a version of the “reform” which all the experts are talking about. But as even Collins was able to note, the track record of these “experts” is embarrassing, awful.

No one’s track record looks worse than Klein’s in the wake of Monday’s news report. But so what! On Sunday, the “Outlook” section put his teacher-trashing “manifesto” right at the top of its page one, next to Miller’s piece. But:

Does Joel Klein know what’s he talking about? Could it be that he has been sent here from Mars?

We’ll consider those questions tomorrow. Quick guesses: No, and perhaps.

Tomorrow: Chancellor Klein’s manifesto

Friday: What ETS doesn’t know