Will NY get real on test cheating?
 |
| Regent Merryl Tisch |
Last Updated:12:03 PM, October 21, 2011
Posted:10:06 PM, October 20, 2011
On Tuesday, the state Board of Regents approved measures to address long-obvious weaknesses in test administration and scoring procedures and to thwart the rigging of results. Yet only one of the steps, at best, deals with cheating -- and that, vaguely and indirectly: a review of how the state Education Department has handled allegations and investigations of improprieties.
There’s still no sign the Regents are ready to go after cheaters and punish them -- from the top down.
Here’s a challenge: If the state wants to show it’s serious about this issue, it ought to probe the results of 4th graders in New York City on this year’s English Language Arts exams.
I’ve found unusual increases in the Grade 4 ELA results -- the kind that often signal problems inherent in exams or their conduct and scoring.
On these tests, the city went up 5.4 points -- an exceptional improvement in one year, considering 70,000 students are involved. It’s even more surprising when you realize that the rest of the state went down 3 points.
This swing of 8.4 points is particularly suspect because the city contains two-thirds of all Limited English Proficient students in the state.
In two of the city’s school districts, results jumped over 10 percent. These large gains cry out for objective investigation and explanation, especially coming against claims the 2011 exams were more challenging.
I believe the surge was due to lenient teacher scoring of open-ended student responses. Investigators should compare scores on the ELA’s 4th grade multiple-choice items against grades on its essay and short-answer questions. Atypical higher-than-average scores for bilingual-education students and other English Language Learners could also be a tip-off that something odd occurred.
Expect the city to resist scrutiny. In July, when State Education Commissioner John King formed a panel “to review all aspects of the [testing] program,” Chancellor Dennis Walcott’s reaction was that the city went beyond the call of duty to safeguard tests. In fact, the city stopped checking answer sheets for evidence of tampering just when Mayor Bloomberg took office.
Nor did Walcott address a city comptroller’s audit charging that DoE lacks adequate controls over testing -- exposing exams to manipulation.
Last month, the city issued this year’s progress reports, assigning letter grades to schools -- 85 percent of which are based on test scores.
Schools can be closed because of low grades. This creates more pressure to cheat and greater urgency for a monitor to delve into peculiar results.
Instead of spending the summer deliberating about “test forensics,” the Regents should have already engaged an outside testing expert to ferret out strange patterns.
Now the Regents want to bar teachers from grading their own students’ tests -- not this year, but next year, eventually closing this score-raising barn door. Why wait?
Regents Chancellor Merryl Tisch said last month, “We need to be absolutely certain that our system is beyond reproach . . . that our tests are not compromised in any way.”
For starters, Albany can back up its rhetoric by supporting an immediate independent investigation of the city’s Grade 4 ELA results.
Fred Smith, a retired Board of Education senior analyst, worked for the city public-school system in test research and development.
Amid Cheating Complaints, Regents Move to Improve Test Security
By SHARON OTTERMAN, NY TIMES, September 12, 2011
But several experts on testing said New York still had a long way to go to be in line with what leading states are doing to prevent educators from tampering with tests. Additional improvements, including some items recommended last week by a state panel, like centralizing scoring in one place and conducting analyses of results, would help, but some local officials worry they could be prohibitively expensive.
“The steps they are suggesting are nothing out of the ordinary,” Gregory Cizek, a test security expert from the University of North Carolina, said of the panel’s recommendations. “They would pretty much get them in line with the ordinary.” He added that New York’s current test-security practices put it “near the bottom” of states nationally.
The Regents on Monday began the process of addressing that, giving state officials approval to explore an additional series of steps that would put the state in control of scoring, a significant shift from the localized systems now in place, which appears to be unique among states.
Scoring all the tests in one place, education officials said, would allow the state to do systemwide checks for cheating, like detecting suspicious patterns of erasures or sudden leaps in scores. For essay questions, state officials will explore a relatively new technology called distributive scoring, in which answer sheets are scanned and uploaded onto computers, and graded by other educators across the state.
“We administer six million assessments a year,” said John B. King Jr., the state education commissioner. “That’s a lot of tests, and that creates a lot of opportunity for things to go wrong.”
The heightened emphasis on test security comes after a spate of cheating scandals, in Atlanta, New Jersey, Philadelphia and Washington. In June, the federal education secretary, Arne Duncan, sent a letter to all state education commissioners urging them to make “assessment security a high priority” by considering additional steps like more monitoring on testing day.
On Monday, the Regents also gave the go-ahead for the state to consider banning teachers from grading or proctoring their own students’ exams, though several members expressed concern that younger students might get too nervous with an unfamiliar adult showing up on test day.
During a half-hour debate, many board members appeared more hesitant than enthusiastic about the scope of the changes. Some worried that a renewed emphasis on preventing cheating might make teachers feel unfairly accused. Others wondered how the state, which canceled some high school exams this year after falling $8 million short in its testing budget, could afford sweeping changes.
“We would not want to set up an apparatus that is so expensive and costly that we distract from our original goal: teaching and learning,” said James O. Jackson, who represents Albany and the surrounding region on the 17-member board.
New York State currently spends about $38 million a year on testing, much of it paid for by the federal government. State officials did not provide a cost estimate for centralizing scoring, saying they would arrive at one in the coming months.
As a general guide, a 2011 report by the Congressional Research Service estimated that scoring of multiple-choice questions alone costs about 15 percent of a state’s testing program (that would be about $6 million in New York). Adding short-answer and essay questions, which the state’s tests have a lot of, would double that cost.
Currently, local districts pay for grading; New York City alone spends about $20 million.
While Dr. King said centralizing scoring would most likely save money over all, there was still concern that districts would end up bearing the burden.
“We’ve been through three financially rough years,” said Bob Lowry, executive director of the New York State Council of School Superintendents. “It’s not a bad idea, but who is going to pay for setting up the system?”
In an interview, Dr. King said part of his work in the coming months would be to convince the governor’s office and the State Legislature that the centralized scoring, and the additional security it could bring through system checks for suspicious erasures, among other matters, would be worth the investment.
The governor’s office did not respond to a request for comment on Monday.
For now, state officials are looking for someone to investigate shortcomings in how the state handles cheating accusations, preferably someone with impeccable credentials who would be willing to take on the project pro bono.
Some recommendations issued on Tuesday by the panel, which was convened by the state’s Department of Education in July, will take effect immediately; others require approval of the Board of Regents, the state education policy board.
Among those that take effect right away: All state tests must now begin on the same day in districts throughout the state, to heighten test security. Also, all educators who proctor or grade state exams will be required to certify that they were trained and followed security procedures.
In outlining the recommendations, John B. King Jr., the state education commissioner, and Valerie Grey, the executive deputy commissioner, wrote in a memorandum to the Board of Regents that not enough was being done to detect and deter cheating by teachers and principals, given the rising stakes of tests.
“Here in New York, as standardized test scores are increasingly utilized for school and district accountability” and in teacher and principal evaluations, they wrote, “it is imperative that those tests are not compromised.”
Some of the deeper changes recommended by the panel would need approval from the Board of Regents. One option is to bar teachers from grading or proctoring their own students’ exams, which has been an accepted practice across the state for decades. The group also recommended that the Board of Regents authorize the state to begin searching for an independent investigator to examine how it currently handles cheating complaints.
The panel said it appeared that New York was the only state that graded its standardized tests locally, a practice that is against federal recommendations. As a result, it directed officials to investigate switching to a statewide system for scoring multiple-choice questions that would include a computer analysis of erasure marks to detect cheating.
For essay questions, the panel recommended that state officials consider a rotation system that would permit all questions to be scored outside the schools in which they are given. To aid in investigations, it recommended that the state consider keeping test answer sheets for longer than a year before destroying them.
 |
| Center: Shael Polakow-Suransky (white shirt) and Former Chancellor Harold Levy |