Poverty has a Powerful Impact on Educational Attainment, or, Don't Trust Ed Trust

Status: 
Archived
Subject: 
K-12 Testing

by Stephen Krashen, Professor Emeritus, University of Southern California. Published in February 2002 Substance (5132 W. Berteau Ave., Chicago, IL 60641; csubstance@aol.com) and reprinted with permission.

 

Two critical responses exposing flaws in a report by Education Trust that claims to have found thousands of schools across the U.S. which serve large numbers of low-income children and which score high on standardized tests. See also FairTest Examiner summary and discussion (Winter 2001-02). Richard Rothstein also critiqued the Ed Trust report, with very similar results. His column is at
http://www.nytimes.com/2002/04/10/education/10LESS.html?pagewanted=print&pos

 

The powerful impact of poverty on literacy development has been well documented. Children of poverty, in addition to the obvious problems they face, have very little access to reading material ; they have fewer books in the home, inferior public libraries, inferior school libraries,and inferior classroom libraries, (e.g. Duke, 2000; Neuman and Celano, 2001). This means, of course, that they have fewer opportunities to read, and therefore make less progress in developing literacy.

 

The recent report from Educational Trust West (Ali and Jerald, 2001) appears, at first glance, to show that a significant number of children in poverty have overcome this problem. The report claimed to find 3,592 schools in the US that were "high-performing-high poverty" schools. In California alone, there were 355 high-performing-high poverty school. This result was considered sufficient to "dispel the myth" about the relationship between poverty and educational achievement, and was followed by newspaper articles proclaiming that these high-scoring schools can "offer a lesson" (New York Times, December 17, 2001; Los Angeles Daily News, December 16, 2001).

 

The Ed Trust Report deserves another look. It has serious flaws, and, in fact, shows exactly the opposite of what it says it shows.

 

Very few schools qualify. The number of schools classified as high-poverty high-scoring represents about 4% of the nation and state school population. Moreover, a closer look shrinks even this number to considerably. In fact, it shrinks it to nearly zero.

 

It is easy to qualify as high-scoring. A high-performing school was defined as one in which students in ANY grade scored in the upper third of the schools in its own state in EITHER math or reading. Thus, a good performance by one grade level (in some schools only one classroom) on one test can qualify a school as "high performing."

 

Consider the case of California. Of the 355 "high-scoring" schools in California, only 134 were high-scoring in reading. There are 8761 schools in California. This means that about 1.5% qualify as "high-flying schools." Of these 134, 83 managed to qualify because of children in only one grade level! This could be due to the performance of a few students in one classroom, perhaps even those from higher-income families (see below). We are now down to 51 schools, about half of one percent.

 

Scores can be based on students NOT considered high poverty. Ed Trust may claim that a grade in a high poverty school reached the upper 1/3, but not all the children at that grade level were high poverty. Consider the case of fourth graders at the Language Academy, a (magnet) school in San Diego. Academy fourth graders scored in the upper 1/3 of the state in reading, averaging 61. But the subset of economically disadvantaged children (n = 27) scored 42, while the advantaged children (n = 36) averaged 73. Fourth graders at Language Academy were classified as high scoring high poverty not because of the scores of its disadvantaged children but because of the scores of its advantaged children. Ed Trust does not present this kind of a breakdown of scores.

 

Ed Trust used a low standard for classification as "high poverty." A high-poverty school was defined as one in which at least 50% of the students were from low-income families. The California average is 46%.

 

The report has numerous inaccuracies. For California, several schools listed as high-poverty were not, and in many cases grade levels Ed Trust said were high scoring were not. The alternative analysis below presents details, as well as confirming that the number of "high-poverty high-scoring schools" is very very small.

 

An alternative analysis

If we define truly exceptional schools as those with at least three grade levels scoring in the upper one-third in reading, we are down to 20 schools in California. Let's take a closer look at the 20: In two cases, the schools did not qualify as high-poverty, even according to the very modest standard set by Ed Trust. 1 For the other 18, a look at SAT9 scores shows that only four of the schools actually had all three classes in the upper one-third in reading, based on California's standards, and none qualified as a high-scoring school using national standards. Of the four that qualified in California, one was a magnet school. The high-scoring classes in the three other schools had a total of 391 children. In one, the Steinbeck school, high scorers in two grades (3 and 6) scored much lower on the language portion of the SAT9 (36 and 30). 2

 

Poverty has a powerful effect

The Ed Trust report is actually a stunning confirmation of the overwhelming effect of poverty. Even with a very loose definition of high performance, few schools perform in the upper one-third and a careful look at one state reveals that even fewer qualify. California has about five million children in school. Ed Trust claimed that about 230,000 were in high-poverty high-scoring schools for reading. According to this analysis, the real figure is less than 400. It is extremely difficult to "defy the odds." Poverty has a powerful effect on educational attainment.

 

Notes

1. The Raoul Wallenberg school reported only 41.5% and Richmond only 36.4% of their students on free or reduced price lunch. Wheatland Union had 50.5% and Pescadero had 50.8% on free and reduced lunch. These were included as "high-poverty" schools.
2. It is a lot easier to place in the upper 1/3 in California than in the most other states; California ranks at the bottom in reading among states in the USA. State averages are really low in grade 2 (30th percentile), 9 (33rd), 10 (33rd) and 11 (37th). The fifth, sixth and eighth grade CA average is 43. All are under the national average of 50.

 

Even using this lower standard, only four schools in California had three grade levels that actually scored in the upper 1/3 for reading: Borrego Springs (81 children), Bravo Magnet (about 1000 children), Steinbeck (193 children) and Kernville (117). Three out of four grades nominated by Ed Trust actually qualified at Steinbeck and Kernville.
For Kernville, grades 4, 5 and 6 met the standard, but there were few disadvantaged children in grade 5.

 

Ten of the 18 schools had no grade levels meeting the California standard for the upper 1/3: Costano, Cottonwood, Florence, Happy Camp High, Hayfork High, Kernville, Muir, Surprise Valley, Surprise Valley High, Van Duzen.

 

In five schools, results were mixed: At Clairemont, grades 9 and 10 qualified, but grade 11 did not. For the Language Academy, grades 3 and 4 qualified, but not the subset of disadvantaged children. Grade 7 did not. For Pescadero, two of the three grades did not qualify. Grade 5 did, but not the subset of disadvantaged children. For Perry: grade 3 qualified but not grades 5 and 6. At Wheatland, grade 9 qualified, but not the subset of disadvantaged children. Grade 10 qualified but grade 11 did not.

 

(California scores were calculated from mean scores (percentile ranks) provided by the State of California Department of Education website, and converting to NCE's.)

 

Here are SAT9 scores for those grades in "high-poverty" schools categorized as achieving in the upper 1/3.

 

1) Borrego Springs; grade 9 = 60; grade 10 = 44; grade 11 = 46; only nine disadvantaged students were tested in grade 9
2) Bravo Magnet: grade 9 = 43 (493); grade 10 = 43 (389); grade 11 = 36 (400); for disadvantaged students only, grade 9 = 44 (434); grade 10= 43 (314); grade 11 = 34 (320).
3) Clairemont; grade 9 = 43 (493); grade 43 (389) ;grade 11 = 36 (400). For disadvantaged students, grade 9 = 44 (434); grade 10 = 43 (314); grade 11 = 34 (320).
4) Costano; grade 3 = 60 (37); grade 4 = 41 (72); grade 7 = 39 (49); for disadvantaged children only, grade 3 = 63 (40); grade 4 = 39 (41); grade 7 = 30 (34). Note that this school reported more disadvantaged children tested than total children tested for grade 3.
5) Cottonwood; grade 3 = 42 (28); grade 5 = 33 (24); grade 7 = 41 (19); for disadvantaged children only, grade 3 = 23 (19); grade 5 = 43 (15); grade 7, no score reported, 10 tested.
6) Florence; grade 3 = 39 (26); grade 4 = 30 (23), grade 5 = 31 (23), grade 6 = 39 (13); for disadvantaged only, grade 3 = 33 (16); grade 4 = 79 (21); grade 5 = 33 (13); grade 6 = 61 (12). Note that the grade 4 and grade 6 scores are mathematically impossible.
7) Happy Camp High; grade 9 = 37 (24); grade 10 = 34 (33); grade 11 = 33 (19). All students were disadvantaged.
8) Hayfork High; grade 9 = 23 (43 students tested); grade 10 = 41 (36 tested); grade 11 = 43 (33 tested); for disadvantaged students only; grade 9 = 30 (23); grade 10 = 22 (23); grade 11 = 24 (32)
9) Kernville; grade 3 = 37 (28 students tested); grade 4 = 70 (31 tested); grade 5 = 73 (23); grade 6 = 63 (30 tested); for disadvantaged students only, grade 3 = 23 (13 tested); grade 4 = 66 (13 tested); grade 5 = no score given, 10 tested; grade 6 = 60 (16)
10) Language Academy (San Diego): grade 3 = 62 (34); grade 4 = 61 (63); grade 7 = 43 (21); for disadvantaged children only: grade 3 = 34 (44); grade 4 = 42 (27); grade 7 = 42 (13)
11) Muir; grade 4 = 43 (26); grade 8 = 43 (34); grade 11, not reported, only ten students tested.
For disadvantaged children only, grade 4 = 37 (130; grade 8 = 41 (23); no scores reported for grade 11, only 3 children tested.
12) Pescadero: This school has 50.8% disadvantaged children. grade 3 = 20 (39 students tested); grade 4 = 37 (28 tested); grade 5 = 60 (21 tested); disadvantaged only, grade 3 = no scores given, 9 tested; grade 4 = no scores given, 3 tested; grade 5 = 12 (23 tested).
13) Perry: grade 3 = 68 (31); grade 4 = 39 (43); grade 5 = 43 (43); grade 6 = 43 (32); for disadvantage children only, grade 3 = 74 (33), grade 4 = 60 (33); grade 5 = 43 (29); grade 6 = 30 (31).
14) Steinbeck; grade 3 = 69 (34), grade 4 = 61 (60); grade 5 = 32 (30), grade 6 = 67 (63). All children were disadvantaged.
15) Surprise Valley; grade 4 = 38 (14); grade 7 = 39 (16), grade 8 = 33 (16). No scores reported for disadvantaged children. Only 9 tested in grade 4, 3 tested in grade 7, 10 tested in grade 8.
16) Surprise Valley High School; grade 9 = 37 (13); grade 10 = 30 (14), no scores reported for grade 11, only 10 students tested. No scores reported for disadvantaged students; grade 9 had 7, grade 10 had 6 and grade 11 had 2.
17) Van Duzen; grade 4 = 44 (13); grade 5 = 38 (17); grade 8 = 30 (11); no scores reported for disadvantage children. Ten tested in grades 4,5 and 3 in grade 8.
18) Wheatland Union High School; grade 9 = 44 (179); grade 10 = 40 (133); grade 11 = 42 (147); for disadvantaged children, grade 9 = 33 (61); grade 10 = 40 (32); grade 11 = 36 (36)

 

Acknowledgment: My thanks to Gerald Coles for numerous insights and suggestions.

 

References

Ali, R. and Jerald, C. 2001. Dispelling the Myth in California: Preliminary Findings from a State and Nationwide Analysis of "High-Flying" Schools. The Education Trust - West. www.edtrust.org

 

Neuman, S. and Celano, D. 2001. Access to print in low-income and middle-income communities. Reading Research Quarterly 36(1): 8-26.

 

Duke, N. 2000. For the rich it's richer: Print experiences and environments offered to children in very low- and very high-socioeconomic status first-grade classrooms. American Educational Research Journal 37(2): 441-478.

 

Los Angeles Daily News. 2001. No more excuses. Dec. 16, 2001

 

New York Times. 2001. School defies the odds and offers a lesson. Dec. 17, 2001

 

Questionable Selections in Massachusetts

In response to Dr. Krashen's study, a FairTest researcher took a quick look at the schools the Education Trust listed for Massachusetts, reporting:

 

There are only 13 schools in Massachusetts cited by the Education Trust cited for the state. Here's what I notice from a quick look:

 

- Two are "selective" schools: O'Bryant is one of Boston's exam schools, so students are there in large part because they already have higher test scores than other city students; Worcester's University Park is a joint venture of Worcester Public Schools and Clark University, with area students who meet particular criteria eligible to attend.

 

Also per the Loveless comment in the Education Week article (January 9, 2002; "Group Cites Needy But High-Performing Schools, www.edweek.org), many schools test small numbers of students, making single-year score gains poor measures of improvement.

 

- At least one school -- Boston's Mary Lyon -- tests so few students that I'm not sure how "improvements" would even be measured. According to MCAS data, it tested 12 students in 2000 and 2 students in 2001.

 

- Also quite small are Fall River's Harriet Healy (testing 23 in both 2000 and 2010; Boston's Bradley (testing 51 in 2000, 58 in 2001) and Warren-Prescott (testing 51 in 2000, 62 in 2001); Worcester's Canterbury (testing 55 in 2000, 30 in 2001) and Heard Street (testing 30 in 2000,
38 in 2001). Talmadge School in Springfield tested 54 students in 2000, 52 in 2001, with "failing" rates going from 0% in English and 11% in math in 2000 to 13% in English and 22% in math in 2001.

 

- Everett's Centre School tested 60 in 2000, 73 in 2001.

 

- Two charter schools also test small numbers: Academy of the Pacific Rim tested 40 eighth graders in 2000 and 50 in 2001, and 14 tenth graders in 2001 (and it's worth noting that a recent Ed Week article profiling the school noted that students could be asked to leave if they
did not meet standards). Lawrence's Community Day Charter School tested 35 fourth graders in 2000 and 29 in 2001, and 16 8th graders in 2000 and 17 in 2001 (and compared with the district, the school enrolls low-income, Latino, and bilingual students at a lower rate, white students at twice the rate, and no students with disabilities).

 

In short, 11 of the 13 named schools are selective and/or test fewer than 60 students, too few in any one year to determine with any confidence whether the results are a one-year fluke or a stable situation. I did not, unlike Dr. Krashen, consider how many grades were tested or used in making the "high flying" determination, or the exact proportions of students in poverty.

 

Ed Trust may have used MCAS scores or local scores for this report. If the latter, the test is not necessarily the same from one district to another. Boston uses the Stanford 9, and Springfield, I believe, has used Iowas. I don't know about Fall River, Worcester, Lawrence, Everett, or the charter schools.