Despite many unresolved technical and equity-related problems, test-makers are plunging headlong into new computerized methods of administering multiple-choice exams. Unfortunately, simply automating bad tests does nothing to solve their long-standing problems and may actually compound them.
The Educational Testing Service (ETS), for example, introduced its computerized Graduate Record Exam in October 1992, and has begun working on a computerized SAT. The National Council of Architectural Registration Boards already uses an ETS manufactured computerized test as part of its professional licensing process. Computerized tests are also part of ETS new generation of teacher tests, the Praxis Series, and the National Council Licensure Examination for nurses is only available on computer. In addition, many colleges, universities, and public school districts are using computerized placement tests.
Proponents of computerized testing praise its ability to provide immediate score reporting and faster and more flexible test scheduling (a test can be offered several times a week rather than a few times per year). They also point to the efficiency of Computerized Adaptive Testing (CAT), which provides a final score with fewer items in a shorter period of time than conventional paper-and-pencil, multiple-choice tests. In the future, computerized testing may also offer a greater scope of types of items through the use of graphics and video. But the new tests are being rushed into operation before adequate evidence of either their comparability to current exams or their fairness have been collected.
Unresolved Problems
Test-makers claims that the scores of computerized and pencil-and-paper tests are equivalent are inadequately supported. In fact, research studies find there usually is a difference. Most studies show higher scores for paper-and-pencil exams, but a few have found advantages for those who take computerized tests. These averages may mask individual variations. Some respondents may still get lower scores even if the average score increases. Also, some types of questions perform differently on the two types of tests.1
Computerized tests constrain test-takers compared to paper-and-pencil tests. With computerized versions, test-takers are unable to underline text, scratch out eliminated choices and work out math problems — all commonly-used strategies. Studies also suggest that computer screens take longer to read than printed materials, and that it is more difficult to detect errors on computer screens.1
Most computerized tests show only one item on the screen at a time, preventing test-takers from easily checking previous items and the pattern of their responses, two other practices known to help test-takers. Scrolling through multiple screens does not allow side-by-side comparisons.
Test-takers with the ability to manipulate computer keys rapidly may be favored by long passages that require reading through many screens.
Test-makers may try to use computerizd exams to circumvent Truth-in-Testing disclosure requirements. ETS has not revealed how it intends to continue making test questions and answers available to university admissions test-takers.
Computers may worsen test bias. The performance gap which already exists on multiple-choice tests between men and women, ethnic groups, and persons from different socioeconomic backgrounds could widen as a result of computerized testing.
Schools with large minority or low-income populations are far less likely to have computers, and poor and minority children are much less likely to have computers at home 2,3. White students are more likely to have earned computer science credit than either African American or Hispanic students3.
The additional cost of computerized tests is certain to have a large effect on who chooses to take them. Poorer students are unlikely to take the computerized GRE, for example, because it costs nearly twice as much as the paper-and-pencil version.
Girls may be adversely affected by computerized tests. A much greater number of females than males report no school access to computers, no computer learning experiences, and limited knowledge about computers3. In addition, computer anxiety is much more prevalent among females than males4, with Black females reporting the greatest anxiety5.
SOURCES:
1 Bugbee, A.C. & Bernt, F.M. Testing By Computer: Findings in Six Years of Use 1982-1988, Journal of Research on Computing in Education (Vol. 23, #1, pp. 87-100, 1990).
2 Sutton, R. Equity and Computers in the Schools: A Decade of Research, Review of Educational Research (Vol. 61, #4, pp. 475-503, 1991).
3 Urban, C.M. Inequities in Computer Education Due to Gender, Race, and Socioeconomic Status, (Exit Project, Indiana University, 1986).
4 Moe, K.C. & Johnson, M.F. Participants Reactions to Computerized Testing, Journal of Educational Computing Research (Vol. 4, #1, pp. 79-86, 1988).
5 Legg, S.M. & Buhr, D.C. Computerized Adaptive Testing with Different Groups, Educational Measurement: Issues and Practice (Summer, 1992, pp. 23-7).