State-run tests are sometimes easy: Many states run testing programs which report the number of kids who are proficient on a given grade level. But proficiency is in the eye of the beholder; a state can set the bar for proficiency as high or as low as it wants. How easy are some state-run programs? Winerip notes the contrast between results on some state-run programs and results on the federally-administered NAEP (the National Assessment of Educational Progress, known as The Nations Report Card):
WINERIP (11/2/05): Take Florida, where 30 percent of fourth graders were proficient in reading on this federal test in 2005. Yet on the Florida state test, 71 percent of fourth graders were proficient in reading in 2005. It's a big difference: Are nearly three-quarters of your fourth graders proficient? Or less than a third? And it's typical.As Winerip notes, the gap is similar in New York—and even wider in Tennessee. Where do you set the bar for proficiency? That is always a matter of judgment—and states can set it as low as they want. Citizens need to keep this in mind when they get pleasing scores from a state-run program. Why, other journalists might even consider this point—if it isnt too taxing, too boring, too depressing, too awkward or just too much trouble, of course.
Uh-oh! Trends may differ: Uh-oh! Sometimes, kids get smarter on a state test—and dumber on the federal test simultaneously. Winerip offers examples:
WINERIP: [B]asic trends on the two sets of tests are often contradictory. In Florida, the federal fourth-grade reading proficiency scores were down two percentage points between 2003 and 2005 (bad news); on the state test they were up 11 points (good news). In New York, on the federal test, fourth-grade reading proficiency was down one point; on the state test, up six points. In short, it's hard to answer the age-old question: Are fourth graders getting smarter or dumber?Minor differences may be statistical blips. But sometimes, kids improve on state tests because the states keep making their state tests easier. This produces a jump in the scores. But the score gains reflect a change in the tests, not in real student achievement.
As weve often noted, Winerip does superlative work at the Times. Here at THE HOWLER, both these points will be in play in the endless but seminal series we brazenly kick off today.
Special report: Making schools (seem to) work!
PART 1—AN IMPORTANT TOPIC: Few topics could be more important. When he introduced his recent two-hour special, Hedrick Smith said hed tell a surprising story—a story of educational success:
SMITH (10/5/05): Good evening, Im Hedrick Smith.The program was called Making Schools Work with Hedrick Smith. It aired on many major PBS stations in the first week in October.
This is an often surprising story of educational success. Because these are public schools—some of them charter schools—serving our most disadvantaged kids, who many had given up on.
Who the heck is Hedrick Smith? As the bio on his programs web site explains, Smith is a Pulitzer Prize-winning former New York Times reporter and an Emmy Award-winning producer/correspondent. Indeed, Making Schools Work is the sixth prime time miniseries that [Smith] has created for the PBS national program service, the site notes. In short, Smith is a man with a major resumé—and PBS is often considered our smartest news service. As he continued, he said a bit more about what his new program would show:
SMITH (continuing directly): Our story begins where reform began in the 1980s, with models designed to make change school by school. Then we move, as reformers moved in the 1990s, to wholesale reform—to re-engineering entire school districts.But has this wholesale reform really worked? Did it produce results in those districts? As he continued, Smith clearly seemed to say that it had. We wont find one magic formula, he said. But we would see a common denominator—results, lifting scores and closing achievement gaps not just for a few hundred kids but for nearly two million children from our inner cities to rural America. Indeed, on the front page of the programs web site, the claims for this PBS show are profound:
MAKING SCHOOLS WORK WEB SITE: No topic worries American families more than the quality of our schools. Making Schools Work with Hedrick Smith offers a rare and often surprising look at success in unexpected places, with enormous implications for public schools nationwide.How enormous are the implications? This web site follows our production team into classrooms from coast to coast to see how some American communities are creating a small revolution in our schools, the text continues. And again: The common denominator is results—lifting scores and closing achievement gaps, not just for a few hundred children but for nearly two million, from our inner cities to rural America.
Smith and his web site were making large claims. His program would have enormous implications for public schools nationwide—indeed, it would show us a small revolution in our schools, a revolution which has already affected nearly two million low-income kids. And make no mistake—claims like this are always pleasing. Over at the Bergen Record, one scribe—Mary Ellen Schoonmaker—swallowed the story line whole:
SCHOONMAKER (10/6/05): Last night on public television, the two-hour documentary "Making Schools Work, with Hedrick Smith" showcased schools in urban and rural settings around the country where low-income students are succeeding. The theme of the program was there are no excuses. Even troubled kids and failing students can turn their lives around. Test scores can go up. And learning can take place.After describing a few of the highlighted schools, Schoonmaker spoon-fed her readers the message. The common thread in all these approaches is confidence, which is contagious, she wrote. Teachers and principals have a plan. They believe in what they are doing because they know it works, and their energy and enthusiasm rub off on the kids.
The strategies for success were all different.
But do the plans at these schools really work? Few questions could be more important. And because the question is so important, we ought to be especially careful in formulating an answer. At the New York Times, Anita Gates had previewed the program—and had offered one cautioning note:
GATES: (10/5/05): ''Making Schools Work,'' narrated by the journalist Hedrick Smith, visits other schools that have had impressive successes, mostly with drastic, all-encompassing programs. The program ends with examples of districtwide reform, including Anthony J. Alvarado's success in District 2 in New York (focusing on Public School 126 in Chinatown) and a disaster of sorts in San Diego.It wont be easy, Gates said as she finished, after citing that oddly upbeat remark about poverty—poverty, which has been deeply and tragically linked, for decades, with low achievement in our schools. But Gates herself never seemed to doubt that the schools on this program had actually had impressive successes due to their all-encompassing programs. Like Schoonmaker, she seemed to believe what she had been told about these featured schools and school districts.
One educator sums it up: ''There is no link between poverty and the ability to learn.'' It is all about making quality teaching available. It requires cooperation and high expectations. It won't be easy.
But what if these featured schools and districts havent had impressive successes? What of their impressive successes have been, to some extent, gimmicked up? What if Smith and his team have somehow misjudged their degree of success? What then are we to make of their all-encompassing programs—the programs Smith would extend to the nation? Because Smiths topic is so important, we ought to make sure that his judgments are real. Before we start to promote the programs which have produced these impressive successes, we really ought to try to make sure that the impressive successes are real.
Smith examines a series of schools and districts—schools and districts which serve large numbers of deeply deserving low-income kids. He says theyve achieved a small revolution, with implications that are enormous. But time and again, when we looked at these schools, we seemed to find something different. Un-oh! For those who want to truly believe, there are ways to make schools seem to work. And we often thought we saw wishful thinking—and slippery analyses—driving Smiths claims of successes.
TOMORROW—PART 2: That grade school in Washington has a great staff. But how sure can we be that its working?