Once again, were not making this up. And according to the material Dawson sent, this absurd statistical procedure has been used in Virginia since 2001; presumably, it may have inflated official passing rates at many other schools in the state. If this has occurred, then the state has been systematically defrauding its citizens through this public reporting (accountability!) system.
Have other schools displayed the pattern observed at Maury last year? Have passing rates been inflated statewide by the use of this absurd procedure? For example, have elementary schools in the Norfolk system had their passing rates inflated? As we noted last week, Norfolk won a prestigious national award for urban school systems last year, granted by the Broad Foundation. But did Norfolk win this national prize based on inflated school passing rates? This may be the worlds most obvious question—and its a question we simply cant answer.
The question is obvious, but it cant be answered—because the state of Virginia has now removed its school report cards from public view. (These are the public records posted on-line to inform the public about public schools. The school report cards have been unavailable since at least March 3.) Of course, nothing on-line disappears completely; some readers tell us that they have been able to access some of these records through various machinations. Yesterday, for example, a reader sent us material he had accessed in an Excel file. Uh-oh! We dont have Excel. And the basic question, of course, is this: Why are these records being withheld from normal public inspection?
For the record, this reader also told us that Norfolk elementary schools did display the pattern found at Maury. At these schools, the passing rate which the state is reporting exceeds the actual passing rates observed in grades 3 and 5. At present, of course, we cant confirm this claim, because the state has disabled its public records. We cant examine these public records, because theyve been taken from view.
Here at THE HOWLER, well follow this continuing story under the heading, Yes, Virginia! Last year, Santa Claus gave the Maury School just what it had always wanted—an utterly absurd statistical process which vaulted the school to page one of the Post. Yes, Virginia, there is a Santa—but is there a news org in the state or the nation which is willing to report on this matter?
THE WASHINGTON POSTS SOFT BIGOTRY: Can you say soft bigotry of low expectations? Last year, 27 percent of Maury third-graders passed Virginias state reading test. This was an abysmal passing rate—about one-third the state average. But so what? The Post hailed Maury on its front page, calling it a source of pride. Do low-income children really matter? Or do they exist to be lied about, so the rest of us can feel real good inside? This week, were asking the states news orgs, as youll see in the segment below.
FIRST, THIS UPDATE: This morning, an e-mailer tells us—apparently from Hawaii—that the school report cards can be accessed fairly easily. How easily?
E-MAIL: I don't think you've highlighted the further absurdity of the "front page" for the school scores. Yep, the report is said to be "Temporarily unavailable—thank you for your patience ," but in reality, the report is still there. The links in your previous writings still work, as does: http://www.pen.k12.va.us/VDOE/src/vasrc-reportcard-intropage.shtml (click here)For the record, we didnt highlight this because we didnt know it. (Others had told us that data could be accessed through elaborate efforts.) Of course, no one in Virginia knows it, either; at the state web site, the link to school report cards is still missing, and a message continues to say that the cards are temporarily unavailable. In short, the state is still withholding these data from citizens. It just doesnt seem to have done so competently. (For the record, were not sure that this link has been active for the past several weeks. It did work for us this morning.)
Well report more on this matter tomorrow. But weve taken a quick look at passing rates in reading (third and fifth grades) in the first four elementary schools on Norfolks list. (Bay View, Bowling Park, Camp Allen, Chesterfield—the report for one other school wouldnt come up. Third and fifth grades were the only grades tested.) In all four schools, the passing rate found at the top of the schools report card is higher than the grade-by-grade rates buried in the charts down below. (This is the pattern we found with Maury—the pattern which Dawson explained.) At Chesterfield, for example, 70 percent of third-graders passed the reading test, along with 78 percent of fifth-graders. But at the top of the schools report card, the state presents the overall passing rate for the two grades combined—93 percent! Old saying: You cant get there from here. Grade school math forbids it.
Few schools scored as low as Maury in third-grade reading last year. But this cursory examination suggests that reported passing rates were being inflated around the state. Its time for news orgs to investigate this, as weve been urging—see below.
Continuing story: Yes, Virginia!
OUR E-MAIL TO TWO MAJOR NEWSPAPERS: To state the obvious, major news orgs should try to determine the facts behind this puzzling situation. To all appearances, the state of Virginia has employed an absurd statistical procedure to inflate official school passing rates—to inflate the basic data Virginia uses to fulfill its obligations under No Child Left Behind. No, it simply wasnt true; if Dawsons explanation is accurate, it wasnt true when Virginia claimed that 17 of Maurys 19 third-graders passed the third-grade reading test. But it did make the parents at Maury feel good—and it went to the top of the Posts front page. Behavior like this makes an utter joke of public reporting; defrauds those parents and the states other citizens; and makes it impossible to have a serious discussion about the problems of low-income ed. Do Virginias low-income kids really count? Does this blatant misconduct even matter?
Major news orgs should report this story, and follow the facts where they lead. With that in mind, we sent an e-mail to the Roanoke Times this Saturday. Well present the text of the e-mail below. But here is the list of people to whom we sent the e-mail:
Roanoke TimesOn Sunday, we sent the same e-mail to the following people at the (Norfolk) Virginian-Pilot:
Shawna Morrison, education reporter
Michael Belcher, news editor
Laurence Hammack, public integrity reporter
Jay Conley, education reporter
(Norfolk) Virginian-PilotToday and tomorrow, well send the same e-mail to appropriate people at the Richmond Times-Dispatch and the Washington City Paper—and at some national news orgs as well. If a state has been systematically misinforming its citizens—as part of the No Child Left Behind program!—that is surely a national story. Its also a story when a state disables its public reporting to avoid public scrutiny.
Suzanne Tate, education team leader
Matthew Bowers, education reporter
Deirdre Fernandes, education reporter
Amy Jeter, education reporter
Lauren Roth, education reporter
Philip Walzer, education reporter
Carl Fincke, senior editor, investigative department
Dennis Hartig, editorial page editor
Well continue to post the names of the journalists whom we contact. Here is the text of this weekends e-mail, as it was sent to Tate:
E-MAIL TO SUZANNE TATE, VIRGINIAN-PILOT (3/12/06):By the way, heres the state of the Virginian-Pilots mission statement (see link above): The Virginian-Pilot is proud to serve as the source of news, information and advertising for Hampton Roads, Virginia. With a half-million daily adult readers, The Pilot is...the largest-circulation newspaper in the state of Virginia. Their emphasis. We have a good friend who once worked at this paper. We hope it continues to serve.
Subject line: Possibly fraudulent SOL test scores; entire state reporting system taken down
I thought you might be interested in investigating a puzzling situation at the Virginia DOE. The state has now disabled its entire "school report card" system (it has been down since at least March 3), perhaps in response to some work I've done about a bizarre statistical procedure the state has been using--a procedure which massively inflates the "passing rates" a school achieves on the SOL tests. The disabling of the state's "school report cards" is a puzzling but secondary story. The larger story is the way the state has apparently been computing individual school "passing rates" on the SOLs. (For the record: I have now been told that this procedure inflated passing rates in a substantial number of Norfolk schools, although this is impossible to check since Virginia's school report cards have all been removed from public view.)
The statistical procedure in question is completely absurd, and it seems to have inflated some passing rates in major ways. Example: At Maury Elementary (Alexandria) a 27 percent passing rate in third-grade Reading/Language Arts (5 out of 19) ended up being reported as 17 out of 19. How did that happen? 12 fourth-graders passed the third-grade test, and that number was simply added to the total of third-graders who passed. No, I'm not making this up--and this procedure has apparently been in place since 2001. (Once again: No, I'm not making this up.) This was explained to me twice in e-mails from Monte Dawson, the director of testing in Alexandria, and he sent me the passage from the state technical manual which laid out this procedure. (How absurd is this procedure? As you can see in my on-line posts (URLs below), passing rates can go over 100 percent if schools use this procedure. The technical manual explains what to do then--"cap" the rate at 100! Third time: No--I'm really not making this up.)
I've been working on this topic at dailyhowler.com; the debate began last month with a front-page Washington Post report about Maury (the school was presented as "a study in pride, progress" due to its high passing rates). After I examined the school's internal data, I noted the inflated passing rate; the Post's Jay Mathews responded to my posts in an on-line report on Feb. 28. (Caution: Even in this follow-up report, he seems to be misdescribe the procedures used to generate Maury's passing rates. They are clearly described in the technical material, and Dawson plainly described them in his e-mails, which I have posted.) At any rate, the state's reporting system has now been taken down completely—and this statistical procedure, in place since 2001, was absolutely absurd. I'm at [PHONE NUMBER], or you can check a fairly heavy body of work since mid-February at dailyhowler.com.
Last Friday's post (3/10/06) begins with a basic review of the "story to date." Links to all the basic materials (including Jay Mathews' two reports) are provided at the end of the post. Here is the URL: http://www.dailyhowler.com/dh031006.shtml
I'll repeat one point: The procedure which seems to have been in use is so absurd it defies belief. But it does seem clear that the state has been letting fourth-graders serve as "ringers" on some schools' third-grade tests (ditto with sixth-graders taking fifth-grade tests). And repeat: At the school in question, only 5 of 19 third-graders passed the R/LA test--and it was reported as 17 out of 19. It would be hard to image a more egregious case of defrauding the public through manipulation of test scores.
School report cards have been unavailable for over a week. I hope you'll try to find out why—and examine this underlying matter. Again, I have now been told that passing rates in a substantial number of Norfolk schools were also inflated by this procedure. Did the Broad Foundation know? How about parents of the students at these affected Norfolk schools?
Note: The Columbia Journalism Review did a full profile of my site last year. http://www.cjr.org/issues/2005/1/twomey-howler.asp
P.S. I'm sending copies of this e-mail to staff writers Bowers, Roth, Jeter, Fernandes and Walzer, and to Carl Fincke and Dennis Hartig. Please understand—if this matter is as it seems, this is a very serious abuse of the testing process. This story won't be going away. I hope you will cover it first.
NO, VIRGINIA: No, Suzanne Tate is not the person to whom we spoke on the phone last week. We assume that shes a dedicated scribe. We hope shell pursue this important matter. Shouldnt low-income kids really count?
BASIC LINKS: On February 2, Maury hit the top of the Posts front page. You know what to do—just click here.
We questioned this story the following week, See THE DAILY HOWLER, 2/6/06, then click forward from there.
Post reporter Jay Mathews followed up on February 28. Click here and you can read every word.
We responded all last week—and the state of Virginia has hidden its data. See THE DAILY HOWLER, 3/7/06, for our first installment.