YES, VIRGINIA! Santa came early to schools in Virginia. We’ve ask Suzanne Tate to report: // link // print // previous // next //
THE STORY-TO-DATE CONTINUES: It seems that Christmas came early for Maury Elementary of Alexandria, Virginia last year. In the spring of 2005, only 5 of Maury’s 19 third-graders passed the state’s “Reading/Language Arts” test, a passing rate of 27 percent. (Statewide, 77 percent of third-graders passed. We’ll call this test “reading” from this point on.) But yes, Virginia—there is a Santa Claus! Thanks to bizarre statistical manipulations, the state ended up reporting that 17 of Maury’s 19 third-graders had passed—and Maury was soon at the top of the Washington Post’s front page, hailed as “a study in pride, progress” (full links below). How did five out of 19 become seventeen? How did an abysmal passing rate become a source of community “pride?” Simple—according to Alexandria testing director Monte Dawson, an undisclosed number of Maury fourth-graders also were given the third-grade test. When 12 fourth-graders passed the third-grade test, they were added to the third-grade total. We know, we know, it sounds impossible—but no, we’re really not making this up. Indeed, Dawson sent us a lengthy excerpt, apparently from a technical manual, which outlined the absurd procedure. What do you do when a school’s passing rate exceeds 100? The excerpt even explained that!
Once again, we’re not making this up. And according to the material Dawson sent, this absurd statistical procedure has been used in Virginia since 2001; presumably, it may have inflated official “passing rates” at many other schools in the state. If this has occurred, then the state has been systematically defrauding its citizens through this public reporting (“accountability!”) system.
Have other schools displayed the pattern observed at Maury last year? Have “passing rates” been inflated statewide by the use of this absurd procedure? For example, have elementary schools in the Norfolk system had their passing rates inflated? As we noted last week, Norfolk won a prestigious national award for urban school systems last year, granted by the Broad Foundation. But did Norfolk win this national prize based on inflated school passing rates? This may be the world’s most obvious question—and it’s a question we simply can’t answer.
The question is obvious, but it can’t be answered—because the state of Virginia has now removed its “school report cards” from public view. (These are the public records posted on-line to inform the public about public schools. The “school report cards” have been unavailable since at least March 3.) Of course, nothing on-line disappears completely; some readers tell us that they have been able to access some of these records through various machinations. Yesterday, for example, a reader sent us material he had accessed in an Excel file. Uh-oh! We don’t have Excel. And the basic question, of course, is this: Why are these records being withheld from normal public inspection?
For the record, this reader also told us that Norfolk elementary schools did display the pattern found at Maury. At these schools, the “passing rate” which the state is reporting exceeds the actual passing rates observed in grades 3 and 5. At present, of course, we can’t confirm this claim, because the state has disabled its public records. We can’t examine these public records, because they’ve been taken from view.
Here at THE HOWLER, we’ll follow this continuing story under the heading, “Yes, Virginia!” Last year, Santa Claus gave the Maury School just what it had always wanted—an utterly absurd statistical process which vaulted the school to page one of the Post. Yes, Virginia, there is a Santa—but is there a news org in the state or the nation which is willing to report on this matter?
THE WASHINGTON POST’S SOFT BIGOTRY: Can you say “soft bigotry of low expectations?” Last year, 27 percent of Maury third-graders passed Virginia’s state reading test. This was an abysmal passing rate—about one-third the state average. But so what? The Post hailed Maury on its front page, calling it a source of pride. Do low-income children really matter? Or do they exist to be lied about, so the rest of us can feel real good inside? This week, we’re asking the state’s news orgs, as you’ll see in the segment below.
FIRST, THIS UPDATE: This morning, an e-mailer tells us—apparently from Hawaii—that the school report cards can be accessed fairly easily. How easily?
E-MAIL: I don't think you've highlighted the further absurdity of the "front page" for the school scores. Yep, the report is said to be "Temporarily unavailable—thank you for your patience ," but in reality, the report is still there. The links in your previous writings still work, as does: http://www.pen.k12.va.us/VDOE/src/vasrc-reportcard-intropage.shtml (click here)For the record, we didn’t “highlight” this because we didn’t know it. (Others had told us that data could be accessed through elaborate efforts.) Of course, no one in Virginia knows it, either; at the state web site, the link to school report cards is still missing, and a message continues to say that the cards are “temporarily unavailable.” In short, the state is still withholding these data from citizens. It just doesn’t seem to have done so competently. (For the record, we’re not sure that this link has been active for the past several weeks. It did work for us this morning.)
We’ll report more on this matter tomorrow. But we’ve taken a quick look at passing rates in reading (third and fifth grades) in the first four elementary schools on Norfolk’s list. (Bay View, Bowling Park, Camp Allen, Chesterfield—the report for one other school wouldn’t come up. Third and fifth grades were the only grades tested.) In all four schools, the passing rate found at the top of the school’s report card is higher than the grade-by-grade rates buried in the charts down below. (This is the pattern we found with Maury—the pattern which Dawson explained.) At Chesterfield, for example, 70 percent of third-graders passed the reading test, along with 78 percent of fifth-graders. But at the top of the school’s report card, the state presents the overall passing rate for the two grades combined—93 percent! Old saying: You can’t get there from here. Grade school math forbids it.
Few schools scored as low as Maury in third-grade reading last year. But this cursory examination suggests that reported “passing rates” were being inflated around the state. It’s time for news orgs to investigate this, as we’ve been urging—see below.
Continuing story: Yes, Virginia!
OUR E-MAIL TO TWO MAJOR NEWSPAPERS: To state the obvious, major news orgs should try to determine the facts behind this puzzling situation. To all appearances, the state of Virginia has employed an absurd statistical procedure to inflate official school passing rates—to inflate the basic data Virginia uses to fulfill its obligations under No Child Left Behind. No, it simply wasn’t true; if Dawson’s explanation is accurate, it wasn’t true when Virginia claimed that 17 of Maury’s 19 third-graders passed the third-grade reading test. But it did make the parents at Maury feel good—and it went to the top of the Post’s front page. Behavior like this makes an utter joke of public reporting; defrauds those parents and the state’s other citizens; and makes it impossible to have a serious discussion about the problems of low-income ed. Do Virginia’s low-income kids really count? Does this blatant misconduct even matter?
Major news orgs should report this story, and follow the facts where they lead. With that in mind, we sent an e-mail to the Roanoke Times this Saturday. We’ll present the text of the e-mail below. But here is the list of people to whom we sent the e-mail:
Roanoke TimesOn Sunday, we sent the same e-mail to the following people at the (Norfolk) Virginian-Pilot:
Shawna Morrison, education reporter
Michael Belcher, news editor
Laurence Hammack, public integrity reporter
Jay Conley, education reporter
(Norfolk) Virginian-PilotToday and tomorrow, we’ll send the same e-mail to appropriate people at the Richmond Times-Dispatch and the Washington City Paper—and at some national news orgs as well. If a state has been systematically misinforming its citizens—as part of the No Child Left Behind program!—that is surely a national story. It’s also a story when a state disables its public reporting to avoid public scrutiny.
Suzanne Tate, education team leader
Matthew Bowers, education reporter
Deirdre Fernandes, education reporter
Amy Jeter, education reporter
Lauren Roth, education reporter
Philip Walzer, education reporter
Carl Fincke, senior editor, investigative department
Dennis Hartig, editorial page editor
We’ll continue to post the names of the journalists whom we contact. Here is the text of this weekend’s e-mail, as it was sent to Tate:
E-MAIL TO SUZANNE TATE, VIRGINIAN-PILOT (3/12/06):By the way, here’s the state of the Virginian-Pilot’s mission statement (see link above): “The Virginian-Pilot is proud to serve as the source of news, information and advertising for Hampton Roads, Virginia. With a half-million daily adult readers, The Pilot is...the largest-circulation newspaper in the state of Virginia.” Their emphasis. We have a good friend who once worked at this paper. We hope it continues to serve.Subject line: Possibly fraudulent SOL test scores; entire state reporting system taken down
Ms. Tate:
I thought you might be interested in investigating a puzzling situation at the Virginia DOE. The state has now disabled its entire "school report card" system (it has been down since at least March 3), perhaps in response to some work I've done about a bizarre statistical procedure the state has been using--a procedure which massively inflates the "passing rates" a school achieves on the SOL tests. The disabling of the state's "school report cards" is a puzzling but secondary story. The larger story is the way the state has apparently been computing individual school "passing rates" on the SOLs. (For the record: I have now been told that this procedure inflated passing rates in a substantial number of Norfolk schools, although this is impossible to check since Virginia's school report cards have all been removed from public view.)
The statistical procedure in question is completely absurd, and it seems to have inflated some passing rates in major ways. Example: At Maury Elementary (Alexandria) a 27 percent passing rate in third-grade Reading/Language Arts (5 out of 19) ended up being reported as 17 out of 19. How did that happen? 12 fourth-graders passed the third-grade test, and that number was simply added to the total of third-graders who passed. No, I'm not making this up--and this procedure has apparently been in place since 2001. (Once again: No, I'm not making this up.) This was explained to me twice in e-mails from Monte Dawson, the director of testing in Alexandria, and he sent me the passage from the state technical manual which laid out this procedure. (How absurd is this procedure? As you can see in my on-line posts (URLs below), passing rates can go over 100 percent if schools use this procedure. The technical manual explains what to do then--"cap" the rate at 100! Third time: No--I'm really not making this up.)
I've been working on this topic at dailyhowler.com; the debate began last month with a front-page Washington Post report about Maury (the school was presented as "a study in pride, progress" due to its high passing rates). After I examined the school's internal data, I noted the inflated passing rate; the Post's Jay Mathews responded to my posts in an on-line report on Feb. 28. (Caution: Even in this follow-up report, he seems to be misdescribe the procedures used to generate Maury's passing rates. They are clearly described in the technical material, and Dawson plainly described them in his e-mails, which I have posted.) At any rate, the state's reporting system has now been taken down completely—and this statistical procedure, in place since 2001, was absolutely absurd. I'm at [PHONE NUMBER], or you can check a fairly heavy body of work since mid-February at dailyhowler.com.
Last Friday's post (3/10/06) begins with a basic review of the "story to date." Links to all the basic materials (including Jay Mathews' two reports) are provided at the end of the post. Here is the URL: http://www.dailyhowler.com/dh031006.shtml
I'll repeat one point: The procedure which seems to have been in use is so absurd it defies belief. But it does seem clear that the state has been letting fourth-graders serve as "ringers" on some schools' third-grade tests (ditto with sixth-graders taking fifth-grade tests). And repeat: At the school in question, only 5 of 19 third-graders passed the R/LA test--and it was reported as 17 out of 19. It would be hard to image a more egregious case of defrauding the public through manipulation of test scores.
School report cards have been unavailable for over a week. I hope you'll try to find out why—and examine this underlying matter. Again, I have now been told that passing rates in a substantial number of Norfolk schools were also inflated by this procedure. Did the Broad Foundation know? How about parents of the students at these affected Norfolk schools?
Note: The Columbia Journalism Review did a full profile of my site last year. http://www.cjr.org/issues/2005/1/twomey-howler.asp
Bob Somerby
[PHONE NUMBER]P.S. I'm sending copies of this e-mail to staff writers Bowers, Roth, Jeter, Fernandes and Walzer, and to Carl Fincke and Dennis Hartig. Please understand—if this matter is as it seems, this is a very serious abuse of the testing process. This story won't be going away. I hope you will cover it first.
NO, VIRGINIA: No, Suzanne Tate is not the person to whom we spoke on the phone last week. We assume that she’s a dedicated scribe. We hope she’ll pursue this important matter. Shouldn’t low-income kids really count?
BASIC LINKS: On February 2, Maury hit the top of the Post’s front page. You know what to do—just click here.
We questioned this story the following week, See THE DAILY HOWLER, 2/6/06, then click forward from there.
Post reporter Jay Mathews followed up on February 28. Click here and you can read every word.
We responded all last week—and the state of Virginia has hidden its data. See THE DAILY HOWLER, 3/7/06, for our first installment.