DO KIPP SCHOOLS RULE? DO PUBLIC SCHOOLS DROOL? Do kids learn more in private schools? Yesterday, we discussed a study which suggests that they dont, not as a general matter (see THE DAILY HOWLER, 1/30/06). But in this mornings Washington Post, Jay Mathews discusses a set of well-regarded charter schools whose test scores tend to be very impressive. These are the KIPP schools (Knowledge is Power Program)—a nationwide set of small, grade 5-8 urban middle schools which create a highly novel school culture. In this passage, Mathews refers to Mike Feinberg, and Dave Levin, who founded the schools in 1994:
MATHEWS (1/31/06): [T]hey fashioned a system of nine-hour school days with extra pay for teachers, an emphasis on character, behavior and students' future in college, and Saturday classes. The program included teacher visits to student homes, mandatory summer school, a requirement for students to call teachers at night if they had homework questions and an elaborate system of student sanctions and rewards, including a year-end trip to some other part of the country.Nine-hour schools days? Saturday classes? Mandatory summer school? Phone calls to teachers? All of this helped produce the highest test scores among middle schools in the Houston and South Bronx areas, Mathews writes. Do kids learn more in these (urban) charter schools? From the following passage, the answer may seem clear:
MATHEWS (1/31/06): Students on average are at the 28th percentile in reading and math on national standardized tests when they enter KIPP. The first five KIPP schools in the country, including [Susan] Schaeffler's KIPP DC: KEY Academy, show students rising to the 74th percentile by the end of eighth grade, according to figures supplied by the San Francisco-based KIPP Foundation.KIPP kids enter at the 28th percentile—and four years later, they leave at the 74th. The success of this rigorous program seems clear. As always, though, there can be problems when we try to judge and interpret such statements.
This passage seems to answer a crucial question: Is KIPP doing well with typical urban kids? Or do its schools attract kids who are more capable—and more motivated—than the mass of kids who attend public schools? Yesterday, we saw what that new study showed—that, while private school kids tend to score better on tests, the advantage tends to disappear when we adjust for income and other family characteristics. But here we see an impressive statistic; KIPP kids only score at the 28th percentile when they enter (in fifth grade)—but they score much higher by the time they finish. This suggests that KIPP kids are no great shakes when they enter, but are doing much better when they leave.
But uh-oh! There may be a problem with that statistic; it may be comparing apples to kumquats. According to KIPP, its graduating eighth-grade students test at the 74th percentile. But did those same kids test at the 28th percentile when they entered in the fifth grade? Or was that the average score for all entering KIPPsters, some of whom may have left the program by the time those eighth grade scores are compiled? If we want to compare apples to apples, we have to compare the test scores of emerging eighth graders with their own test scores from the time that they entered. It isnt clear that this statistic does that. But so it often annoyingly goes when we try to evaluate claims made by schools and school programs.
KIPP seems to be achieving good test scores. But are its kids really run-of-the-mill, or is the program achieving these score with a higher grade of student? Later, Mathews presents two conflicting views on that basic question:
MATHEWS (1/31/06): Looking at four KIPP schools, Columbia University Teachers College researchers Richard Rothstein and Rebecca Jacobsen concluded that students starting the program in fifth grade had more motivated parents and better test scores than their community averages. KIPP officials said their data showed no significant difference in academic skills between their entering students and other nearby children.Given its rigorous program (and its teachers dedication), we have always assumed that KIPP might well produce above-average student achievement. But here again, we see the question that must always be asked when analyses like this are offered. To what extent does a program like KIPP attract a more motivated, more capable type of student? In the study we looked at yesterday, researchers found that public schools did better than private schools when such matters were factored in. This basic question must always be asked when programs are reviewed in this way.
SADLY, ANOTHER QUESTION: Sadly, another question must always be asked in these matters—is the testing on the level? Over the course of the past forty years, schools and school systems have endlessly cheated to produce high scores on high-stakes exams. (And yes, were talking about cheating here—not about teaching to the test.) Sometimes, this is done deliberately; in other cases, well-meaning educators may engage in bogus practices without understanding that their conduct is wrong. For ourselves, weve always assumed that KIPP is on the side of the angels; well continue to think so until were shown otherwise. But heres a passage about that DC KEY Academy (see above) which simply has to give pause:
MATHEWS (1/31/06): [Its] first class of D.C. students, all black and 84 percent from low-income households, had average math scores that went from the 34th percentile when the students entered fifth grade in 2001 to the 92nd percentile when they completed eighth grade last year, and were the highest in the city last year at the school...Forgive us our suspicious mind, which we acquired through long experience. But if that means that the average eighth-grader at this low-income school was scoring at the 92nd percentile, wed want to see that result replicated—replicated by a set of tests conducted by outside professionals.
Is it possible—is it imaginable—that some KIPP schools could, in some way, be screwing up their testing procedures? Oddly, Mathews doesnt raise the question today. We say oddly because of an excellent piece which Mathews wrote just two weeks ago. That on-line piece was also about the KIPP schools—and Mathews did raise this delicate question, as was completely appropriate. In a long, thoughtful passage about KIPPs test scores, Mathews noted that judging schools is a very tricky business. Then he offered this remark about that jump in test scores:
MATHEWS (1/17/06): Among the most important things to say, for instance, about that astounding jump in math and reading achievement from the 28th to the 74th percentile is that it is based not on an independent study but on standardized tests that KIPP teachers gave to their own students. KIPP schools are run by their mostly young principals, not by the KIPP Foundation, and the principals decide which tests to use and conduct the testing. The schools in the report gave the Stanford 9 or Stanford 10 tests or the Iowa Test of Basic Skills.High praise to Mathews for obliquely noting, in that last sentence, that principals sometimes overtly cheat on such high-stakes tests. For the record, the fact that its the Stanford 9 or the Iowa Test has nothing to do with the issue at hand. Its especially easy to cheat on such tests, whose items dont change from year to year, or to invalidate them unknowingly; indeed, this has happened again and again (and again and again), in documented cases all over the country. Good for Mathews! In raising this caution flag, he did something education scribes rarely do. His piece today would have been much stronger if this concern had not been dropped.
These are popular tests used by many school districts to see how their students compare to a national sample of students, and are as good as you are going to get in the world of relatively inexpensive off-the-shelf tests. But this is still KIPP testing itself, and that should be kept in mind. There are several cases of public school principals changing standardized test papers to enhance their results when they have had an opportunity to do so.
We assume that KIPP is on the side of the angels. We assume that its rigorous program—and the dedication of its teachers—is producing better results than its students would have attained in their neighborhood public schools. But if we actually care about what is true, well trust but verify in such matters. How typical are the kids at KIPP? And can KIPPs high test scores be replicated? We have no idea what the answers may be. But if we care about urban schools, the questions must be asked—every time.
CINDERELLA MAN: For the record, Mathews himself has come a long way since starting out at Hillsdale High. To recall his own Cinderella story, see THE DAILY HOWLER, 9/28/05.
Special report: The Couric Show!
PART 2—RUSSERT, CLEAR AS MUD: How cosmically dumb is our public discourse? To all appearances, Katie Couric was totally clueless when Howard Dean did the Today show last Thursday (see THE DAILY HOWLER, 1/30/06). Dean offered Couric a familiar old point—Jack Abramoff didnt give any money to Dems. By this time, anyone would have known what the Dem chairman meant. But Couric—shes paid millions—was clueless:
COURIC (1/26/06): Hey, wait a second! Democrats took money—Democrats took money from Jack Abramoff, too, Mr. Dean!Pathetic! Dean was talking about Abramoffs personal giving. Couric responded with information about the giving of Abramoffs clients. By this time, any sentient American scribe should have understood this distinction, which had been kicked around for weeks. But to all appearances, Couric—paid millions—didnt understand what Dean meant. She closed the segment with a pitiful promise: Well, we'll obviously have to look into that and clarify that for our viewers.
DEAN: That is absolutely false! That did not happen! Not one dime of money from Jack Abramoff went to any Repub—Democrat at any time.
COURIC: According—let me just tell you. According to the Center for Responsive Politics, Abramoff and his associates gave $3 million to Republican and one—Republicans—and $1.5 million to Democrats including Senate Minority Leader Harry Reid.
Good grief! But then, how dumb is Americas discourse? The next day, Tim Russert showed up to clarify—and yes, the dumbness dragged on. Here was the initial exchange when Matt Lauer raised the confounding conundrum, the one that had even stumped Katie:
LAUER (1/27/06): Let's talk about the Abramoff scandal. Howard Dean was on this program yesterday and asserted basically that it is a Republican scandal. Let me play you a clip.Many observers have chided Lauer for saying that Dean was just technically correct. We wouldnt have phrased it that way ourselves, but were less upset than these others have been, because Dean was flogging a needlessly narrow point—a point he made no effort to clarify (more on that in Parts 3 and 4). For ourselves, well focus on those Center for Responsive Politics numbers—the same data Couric had cited on Thursday. To our mind, Lauers citation was just as dumb as Courics original cluelessness.
DEAN (videotape): It is a Republican-financed scandal. Not one dime of money from Jack Abramoff ever went to any Democrat. Not one dime.
LAUER: And Katie pressed him on that, and then we, we did some research. We went to the Center for Responsive Politics and we found out that, technically speaking, Howard Dean may be correct. But here's what we found. That 66 percent of the money in this situation went to Republicans, but 34 percent of the money—not from Abramoff, but from his associates and clients—went to Democrats. So, can Democrats wash their hands of this?
What was wrong with Lauers presentation? Just this: There is absolutely nothing wrong with accepting donations from Abramoffs clients! True, our system is built on conflict of interest, but thats the way our campaign system works; a wide range of individuals and groups donate money to various pols every single day, and there is no assumption that something is wrong with such contributions until some such showing is actually offered. Did Abramoffs clients donate $3 million to Republican pols, and $1.5 million to Dems? There is nothing wrong—or scandalous—about such donations until some specific showing is offered. Until some specific showing is made, theres nothing wrong with this money being given to Dems—and theres nothing wrong with the money being given to Reps! Groups make donations like this every day. Earth to Lauer: This is how our campaign system works!
Will it turn out that something was wrong with some of these campaign contributions? Prosecutors seem to be interested in some of these contributions, suspecting a quid pro quo was involved—but in the absence of such a finding, there is no apparent wrongdoing with the vast bulk of these gfits. Just what is the point of listing these data? More specifically, what must Democrats wash their hands of? On Thursday, Couric was weak on her facts. One day later, Matt was slow on his logic.
What exactly is supposed to be wrong with accepting donations from an Indian tribe? Answer: There has been no finding that anything was wrong with the vast majority of these donations. But Lauer—like many scribes before him—ran straight to these irrelevant data as if they somehow represented a string of troubling guilty pleas. And Russert failed to offer much clarity. Instead, he just mouthed a press script:
LAUER: So, can Democrats wash their hands of this?By this time, all was clear as mud—just as our press corps constantly leaves it. Our question: Democrats get raging mad when you suggest that what is a bipartisan scandal? Clearly, Russert suggested that there was a scandal in the receipt of these funds from Abramoffs clients. But what exactly is that scandal? Neither Lauer nor Russert really tried to explain. Instead, the hapless Lauer signed off, saying that wed all be sure to watch Tim on Sunday morning.
RUSSERT: No. They will say it is primarily a Republican scandal because the personal money of Abramoff went only to Republicans. But, Matt, the issue is broad and wide. Democrats also understand that they accept trips from lobbyists and—and meals and so forth, and that's why in order to reform all this, it has to be a bipartisan approach. But Democrats get raging mad when they—you suggest this is a bipartisan scandal. They'll say It's primarily Republican, but we're willing to help clean it up.
Just what is the bipartisan scandal? What kind of scandal did Russert mean? As always, The Couric Show was clear as mud. Can this really be the best that these big TV stars can manage? Or is Courics program, like Trumans show, a bit of a fake, staged event?
NEXT: As always, in thrall to the script.
WE KNOW, WE KNOW: We know, we know—studies have suggested that Abramoffs clients gave more to Republicans once he took over. In some ways, this is hardly surprising; as some may recall, Republicans seized the White House in 2001, and began to consolidate power. But readers! Until some specific showing is made, theres nothing wrong with Abramoffs clients giving that money to Republicans either! Duh! Thats the way our campaign system works. What exactly is the scandal to which these muddled millionaires made reference?
Under our system, none of this giving counts as a scandal—until someone explains why it does. Russert, on The Couric Show, made no real effort to do so. He did adhere to a new press script, as well discuss in Part 3.