Quality Counts Questions Testing

K-12 Testing

Quality Counts 2001, this year's edition of the Education Week annual report on school reform, raises serious questions about the harmful impact of standardized tests on standards-based school reform. The report argues that "without a better balance among standards, tests, and support, the movement [for standards-based reform] could fail."


Based on a survey of more than 1000 teachers; a review of research on the quality of state assessments conducted by Achieve, an organization set up to promote state standards and tests; and other research, Quality Counts says states often put too much weight on their tests. The resulting problems include narrowing curriculum and instruction, too much time on test preparation, tests that fail to adequately assess the standards, and a serious lack of support to enable students to meet the standards or to help those who fail.


Similar to other recent surveys of educators, Quality Counts found that nearly seven in ten teachers said "instruction stresses state tests 'far' or 'somewhat' too much," forcing them to "concentrate too much on what's tested to the detriment of other important topics." Nearly half said they spent "a great deal of time" teaching test-taking skills. A majority of teachers also said standards contributed to positive results such as a more demanding curriculum, higher expectations, more writing by students, and increased teacher collaboration.


In past years' ratings, Quality Counts' editors awarded states points for having a graduation test. Because of the controversy over that issue, Education Week stopped that practice; but it does give points to states which base school rewards or sanctions on test scores. The formula also awards points for having tests that use open-ended or extended-response items, but it does not provide any evaluation of the actual quality of those tests. And it gives points for quality of standards, though it relies on an American Federation of Teachers study which evaluates state standards for clarity and specificity but not for intellectual quality.


Wrong Framework ?
Quality Counts accepts as its framework the as-yet-unproven assumption that standards-based reforms controlled at the state level through test-score-linked rewards and sanctions will lead to lasting school improvement. While it makes note of rising test scores as evidence of success, in fact the independent National Assessment of Educational Progress (itself a limited instrument) generally fails to show significant score gains overall or in states with heavy-handed testing programs.


An analysis by the Wisconsin-based Center for the Study of Jobs & Education reports that the Quality Counts rankings are often inverse to state achievement as measured by NAEP. This is because many states which score highest on NAEP have been reluctant to control curriculum through standards and tests. As Fair-Test has pointed out, the impetus behind the movement has been the experience of southern states, which have generally had the weakest education systems but the most tests and the highest stakes.


The report also assumes that tests which adequately reflect high-quality standards can and will be designed and administered. The reality is that such tests will not be implemented because the cost of scoring and the time taken away from teaching to administer valid tests that produce individual scores would be prohibitive.


In sum, while Quality Counts has done a real service by pointing out some of the key dangers in the over-reliance on testing, the assumptions guiding the annual reports should be re-examined, as should the policies governing most state school reform efforts.


• Quality Counts is available from Education Week, 6935 Arlington Rd. Ste. 100, Bethesda. MD 20814-4233; (310) 280-3250, for $10; and on the web at www.edweek.org.
• FairTest's critique of state tests, State Exams Fail Test of Quality, is on the web at www.fairtest.org/arn.html; or send a self-addressed stamped envelop to State Exams at FairTest.
• Center for the Study of Jobs and Education report is at www.jobseducationwis.org.