Testing Consortium Guidelines Unlikely to Improve Assessments

K-12 Testing

FairTest Examiner, May 2010


he U.S. Department of Education (DOE) has issued guidelines for multi-state consortia to follow in applying for federal funds to design new “comprehensive” assessment systems. Unfortunately, DOE’s guidelines assume No Child Left Behind’s negative features will remain in place. Most of the language suggests DOE wants only incremental changes. Thus, proposed systems are not likely to solve the problems created by NCLB, nor lead toward high quality, educationally sound assessment. Still, consortia could significantly stretch the boundaries with well-crafted proposals.

Probably the biggest public complaint about NCLB is that it narrowed the curriculum. A “relentless,” “laser-like” focus on raising test scores turned reading and math into test-prep programs. At the same time, many schools took away time on untested subjects. The assessment consortia guidelines continue to focus on reading and math, continue annual testing in grades 3-8, continue to test every child instead of using sampling for accountability, and continue to be used as the virtually sole basis for high-stakes decisions. They also continue to prioritize large-scale tests. They fail to encourage – though they do not prohibit - systems that use multiple sources of evidence of student learning, including classroom-based evidence, as recommended in the Forum on Educational Accountability’s assessment report as well as by FairTest.

The Department wants the new assessments to support “growth models” and to be used for measuring the “value” added by teachers. That means using tests for a “significant part” (as Secretary Arne Duncan has put it) of teacher evaluation in reading and math. To avoid holding individual teachers accountable for their students’ summer learning loss, there will have to be tests in both fall and spring – which would double the amount of NCLB-mandated testing. Also, experts have repeatedly warned this technology is not ready for high-stakes uses.

The guidelines allow for use of “formative” and “interim” (“benchmark” or “periodic”) tests as part of assessment systems. However, done correctly, formative assessment (assessment for learning) does not consist of centrally administered small versions of large-scale standardized tests. Rather, teachers use various means to find out what students have and have not grasped and then adapt curriculum and instruction based on that information. Unfortunately, test companies have found a lucrative market for low-quality mini-tests, which they falsely label “formative assessments.” Since these companies may well develop the consortia’s systems, they would likely build in expanded use of their profitable products. 

The guidelines ask that the new assessment systems “measure student knowledge and skills against the full range of the college- and career-ready standards, including the standards against which student achievement has traditionally been difficult to measure” (p. 33), and “elicit complex demonstrations of applications of knowledge and skill” (p. 34). But these skills are not simple to assess – and likely cannot be with a standardized test coupled with perhaps a few standardized performance tasks administered during the year. Evaluating them will require tapping into ongoing classroom-based evidence. It will be up to the Administration to assure that proposed assessment systems comply with these important guidelines.

What’s the best a consortium could do within the Department’s guidelines?

  1. Emphasize high-quality performance tasks and have them constitute most of the score; develop libraries of tasks teachers can use for instruction and assessment purposes; ensure they meet the needs of diverse learners; and provide flexibility in which tasks are used and when teachers use them.
  2. Recognize formative assessments are teaching tools that require flexibility and discretion, and improvement in such assessment is best addressed by professional development.
  3. Give significant weight to local and classroom evidence of learning.
  4. Insist that the evaluation of teachers be based on multiple sources of evidence of student learning, with standardized test results only a small part of that evidence.

As it writes a new Elementary and Secondary Education Act (ESEA) to replace NCLB, Congress should consider how to support development of high-quality systems and overcome the flaws in the DOE guidelines.

Finally, to fulfill Secretary Duncan’s promise of “transparency” in the review process, the material submitted by contestants as well as the names of those who will judge the competition should be made open to the public before decisions are made.

- The guidelines (request for proposals) are available at http://www2.ed.gov/programs/racetothetop-assessment/index.html. In addition to comprehensive assessments, there is a competition for high school course exams, with many similar requirements. While each is dozens of pages long, the assessment content components are a few pages each. Applications are due June 23, 2010.