Federal Standards for Standardized Tests

University Testing

Judging by the howls of anguish from conservative "op. ed." writers and the "crocodile tears" coming from some leaders of the college admissions community, one might think the U.S. Department of Education's Office for Civil Rights (OCR) had proposed banning every standardized test which yielded group differences in scores.


In reality, the draft Education Department Resource Guide, released in late May, does little more than summarize current law, court decisions and professional guidelines concerning test score misuse. Because it is not in the form of regulations or formal guidelines, it probably will have minimal legal clout.


Nothing in the document prohibits use of standardized exams, even if they have large disparate impacts by race or gender. Rather it restates principles quite familiar to those who have followed Title VI (race) or Title IX (gender) bias in education cases and their counterparts in employment testing.


The standard for proper test use, as summarized by OCR, is simple: if an exam has a significant disparate impact on minorities and women, then it is the obligation of the institution requiring it to show that the use of test scores is an "educational necessity," that the exam is technically sound (valid and reliable), and that there is no practical alternative mechanism of equivalent or better quality that achieves the same educational goal with less disparate impact.


The negative response to OCR from right-wing think tanks and law firms which litigate against any manifestation of affirmative action is predictable. They automatically rail against any proposal that undermines the myth that "test scores equal merit" (except, of course, those that benefit their supporters, such as preferential admission for the children of alumni), and they consistently attack proposals that might level the racial playing field.


Representatives of groups that manufacture tests or consider their results in college and graduate school selection have a different problem. Guidelines for proper test usage circulated by the College Board (the SAT's sponsor), ACT (the other major college admissions test maker), and the National Association for College Admissions Counseling (NACAC) all warn against using instruments that are not regularly checked for accuracy and specifically enjoin reliance on test scores as the sole factor in determining admission or awarding scholarships. So, too, do the official standards of the mental measurement profession.


Unfortunately, abuse of these guidelines is widespread. Many universities and scholarship agencies use cut-off scores -- absolute minimum requirements -- to make selection decisions. Few have up-to-date studies justifying their testing requirements. Thus, many in the college admissions world correctly fear that the OCR publication will highlight the conflict between their high-sounding principles and their improper practices. Worse yet, focussing attention on their misuses could stimulate successful civil rights complaints and even costly law suits.


Indeed, FairTest-initiated legal actions against the National Collegiate Athletic Association (Examiner, Spring 1999), the National Merit Scholarship Qualifying Test (Examiner, Fall 1996) and New York State's scholarship program (see Examiner, Spring 1989) already have resulted in decisions striking down systems that use test scores as the sole criterion to make high-stakes decisions. With or without any new guidance from OCR, more such litigation is likely since it is fully supported by federal law and court precedents.


K-12 Testing
While the first wave of public debate has focused on the impact of the OCR Resource Guide on college admissions, it's real, long-term import may be in the K-12 testing arena. A growing number of states use test scores as a sole and automatic factor determining school promotion and graduation, a trend recently condemned by a National Academy of Sciences panel (Examiner, Fall 1998) as violating proper measurement practice. Most of these exams have resulted in large racial gaps in passing rates. Conceivably, concerned parents, civil rights groups and others could point to the OCR publication as evidence in suits challenging any of these high-stakes testing requirements.


That possibility puts OCR and the rest of the Clinton Administration in a political bind as the Resource Guide is revised for official publication. If the final version continues to document the likely illegality of many state exams, key elements in the "school reform" agenda promoted by the President and Education Secretary Richard Riley will have been strongly criticized by an agency under their direct control. But if the language is watered down or eliminated, civil rights groups and testing critics will correctly blast the Administration for weak-kneed hypocrisy.


In fact, the document does need changes -- but to strengthen, not weaken, its influence. The statistical standard for "disparate impact" is vague, and the timeline for investigating complaints is unspecified. Most importantly, sanctions for violations are not defined. If the Clinton Administration were truly serious about rooting out improperly used, discriminatory tests, OCR would publish clear regulations, not just a Resource Guide.


Despite these flaws, OCR is on the right track, focusing on the problems with practices that elevate test score differences between racial groups into barriers to equal opportunity. An appendix to the Resource Guide listing sample questions for evaluating testing practices shows that responsible exam manufacturers and score users have little to fear, but those who chronically abuse these instruments are now on notice: test score misuse violates federal law.


In fact, the fundamental principle which underlies the draft OCR Resource Guide should be applied even more broadly. Just as has long been the case for prescription drugs, test promoters should have to prove that their products are "effective and safe" before they can be administered to the public. That is the best way to keep dangerous educational "snake oil" out of the marketplace.