Test-Based Grade Retention Program Fails in Chicago

K-12 Testing

The first independent study of Chicago’s expensive grade retention program shows that about 70% of the 10,000 students who were required to repeat a grade in 1997 failed to meet minimum test scores for promotion set by the school system, even after test- prep summer school and a year of retention — at an annual cost exceeding $100 million. Retained students gained no more than similar low-achieving Chicago students who had simply been promoted before the new policy was put in place.


Researchers from the Consortium of Chicago School Research, who conducted the study, said it was “still quite early” to make statements about “whether the policy is working.” However, the educational research and reform group Designs for Change (DFC), which carried out an additional analysis of Consortium data and also asked three independent academic experts to review the Consortium report, called for “promotion with intensive help” as an alternative to retention for most students.


Harvard University professor Gary Orfield summed up the independent reviewers’ conclusions: “There is massive evidence that this is a misbegotten policy and tens of thousands of students’ futures are now at stake. We should stop this policy now before there is irreparable damage to a generation of Chicago school children.” The national experts also urged other school districts not to follow Chicago’s lead.


Chicago has used the Iowa Tests of Basic Skills (ITBS) as a sole hurdle for promotion from grades 3, 6 and 8 (see Examiner, Fall 1997) for the past three years. Students who do not achieve a minimum test score in May are required to attend summer school, after which they are re-tested and then retained if they still do not pass.


President Clinton and other leading political figures have praised Chicago for “ending social promotion,” although previous studies have shown conclusively that retention leads to no academic improvement, increases the dropout rate, and emotionally damages children. Using tests as Chicago does contradicts the standards of the measurement profession (see related story), the recommendations of test publishers and the National Research Council’s finding (see related story).


In addition to assessing the Chicago initiative’s effect on retained students, the Consortium researchers attempted to gauge the impact of test-based promotion on students who were promoted -- either because they met the minimum test score at the end of the regular school year or because they met the test standard after attending summer school.


The Consortium study found that 3% more third graders, 7% more sixth graders, and 12% more eighth graders met the ITBS cutoff score at the end of the regular school year in May 1997, as compared with May 1995 before the retention policy was in place. However, while the Consortium researchers underscored such gains during the regular school year in their press materials, their study itself states that any one of five reasons could have led to these regular year test score improvements, including teaching to the test. In 1995, the ITBS had no stakes attached to it, while by 1998, the ITBS was being used not only for promotion decisions, but also to put schools on probation and to evaluate principals. Further, various versions of the test had been used several times and were no longer secure (see related story).


The Consortium study also found that students whose summer school score gains were sufficient to gain promotion to the next grade (slightly less than half of those who attended) returned to a low rate of progress in their next year in school. The Consortium researchers call the summer school test gains “one of the most successful aspects of the policy.” They point out that those students who were furthest behind made the greatest gains during summer school. However, the researchers then state that they are unsure whether summer school students who met the cutoff score increased their academic proficiency over the summer or were simply prepped to take the ITBS.


Professor Robert Hauser of the University of Wisconsin, who headed a committee of the National Research Council (NRC) that specifically examined Chicago’s retention program, states that “the NRC Committee concluded that Chicago’s regular year and summer school curricula were so closely geared to the ITBS that it was impossible to distinguish real subject mastery from mastery of skills and knowledge useful for passing this particular test.” In reviewing the Consortium study, Hauser concluded that Chicago’s scores also reflect the “test-score creep” associated with repeated administration of the same test.


At-Risk Students
The Consortium found that African Americans were most at risk for retention, in part because 62 percent of Latinos are excluded from the promotional policy at grade 3 due to limited English proficiency. At grades 6 and 8, blacks and Latinos are equally at risk of being held back. Overall, DFC found that in the fall of 1997, four percent of white students were retained, as compared with 18 percent of African Americans and 11 percent of Latinos. Such evidence bolsters the racial discrimination complaint against the Chicago Public Schools filed with the U.S. Department of Education (see related story). The city plans to raise the test score requirements this school year, which will only cause many more students, particularly minority children, to fall victim to tests and retention.


In making recommendations for change, the Consortium report argued that test scores should not be sole hurdles for promotion. They pointed out that using grade equivalent (GE) scores to make retention decisions lacks accuracy. With only 48 reading questions at grade 6, GE scores range from 1.9 to 16.3, with 134 distinct GE levels — far more levels than questions. Getting one more item correct can increase a score from .3 to .4 GEs, enough to change a promote/retain decision. The error range at 95 percent probability of accuracy is about 1.8 GEs.


The Consortium also recommended that a different instructional approach be used during the retention year, but did not call for the end to large-scale retention.


DFC, on the other hand, urged a complete overhaul of the district’s promotion policies. Preventing low achievement is the best approach, they say, through such initiatives as preparing all teachers to become effective reading teachers and extending quality early childhood education to younger students. For students who are still not progressing well even with early intervention, said DFC director Don Moore, promotion with intensive special help is preferable to retention, even with help. DFC urges that retention should only be used if a child’s parents request it.


Consortium report at www.consortium-chicago.org; 1313 E. 60th St., Chicago, IL 60637, 773/702-3364


DFC report, with comments from experts, at www.designsforchange.org; 6 N. Michigan Ave., Ste. 1600, Chicago, IL 60602; 312/857-9292.