The "high standards" of the education testing industry. Connection to S&P rating of US credit

[Hannah Bell]

There are those who think American public education is in shambles and needs to be completely remade, arguing that the standardized tests mandated by No Child Left Behind will be a vital tool in turning around U.S. schools: They say student promotion and graduation, teacher pay and employment, and school funding should all be tied to NCLB tests. As someone who spent many years working in the assessment industry -- not to mention someone who has been reading the newspaper for the last ten years -- I can't say I understand that idea....

We know that testing data can be manipulated to tell any story. We know that a school administration -- by making test questions easier or lowering cut scores -- can portray improvement in its classrooms even when such improvement doesn't really exist, as happened most recently in 2009 in the New York City schools. We know that "rogue" teachers or administrators -- by erasing incorrect student answers and changing them to correct ones -- can show student achievement even if there is no such achievement, as scandals in Atlanta and Detroit during 2010 both revealed and the current erasure debacle in Washington, D.C. also seems to show. And we know, from my book, that the testing companies fudge numbers all the time, whether reliability numbers (to show the industry is doing a more "standardized" job than it really is); validity numbers (to show the industry is doing a more accurate job than it really is); or score distribution numbers (when test scoring companies work to ensure student results match the predictions of their own psychometricians).

On top of what seems a general dubiousness about the numbers produced by standardized tests, we also know the testing industry regularly screws up. In the last decade or so, scoring errors have occurred on tests returned to students in Arizona (1999-2000), Washington (2000), Virginia (2005), Florida (2006), South Carolina (2008), and Minnesota (2010), not to mention Indiana, Illinois, Connecticut, etc. In 2000, a scoring error by NCS-Pearson (now Pearson Educational Measurement) led to 8,000 Minnesota students being told they failed a state math test when they did not, in fact, fail it (some of those students weren't able to graduate from high school on time because of it). In 2004, ETS erroneously informed over 4,000 teachers they had failed a PRAXIS exam that they had actually passed, leading to lost jobs and lawsuits aplenty. In 2006, Pearson again erred, giving lower scores than were deserved to more than 4,000 students taking the SAT... Keep in mind, also, that most of those errors were discovered only after a test-taker complained about a score, not when any company voluntarily disclosed the problem, raising questions about the legitimacy of every other test administered over the last 10 years...

Last year, I was amazed to see the management of a publishing company giving its test developers only four weeks to produce K-12 assessments for the Detroit Public Schools (a school system now bankrupt, but then, willing to pay millions to a testing company); later, however, that short time-frame looked like a leisurely vacation compared to breakneck pace the company next worked its employees at, when the test development staff was required to pound out more than 200 Common Core Standard tests over the next two months. Two hundred tests is probably more than a not-for-profit like ETS has developed in its entire history, but in a rush to address the new CCS market and get their hands on "Race to the Top" money that company worked its employees nearly to exhaustion and seemed willing to go to any length to write those tests: They recycled items used many times on previous tests, re-aligned items to link them to academic standards they were only sort of linked to, hired people with neither teaching nor testing experience to work as full-time test developers, employed any consultant off the streets willing to work, and re-hired testing vendors previously fired for the poor quality of their work (one of those vendors celebrated its renewed contract by immediately advertising on Craigslist, hoping to find anyone at all willing to write test questions for $8 each)...

http://www.huffingtonpost.com/todd-farley/standardized-testing-a-de_b_846044.html