A Learner-Centered School Accountability Model: An Alternative to High Stakes Testing

Ken Jones

For some time now, it has been apparent to many in the educational community that state and federal policies intended to develop greater school accountability for the learning of all students have been terribly counter-productive. The use of high-stakes testing of students has been fraught with flawed assumptions, oversimplified understandings of school realities, undemocratic concentration of power, undermining of the teaching profession, and predictably disastrous consequences for our most vulnerable students. Far from the noble ideal of leaving no child behind, current policies, if continued, are bound to increase existing inequities, trivialize schooling, and mislead the public about the quality and promise of public education.

The problems with the existing approach are manifest. Based on the results of a single test, huge numbers of schools have been declared failures in a time when public surveys indicate very high public approval ratings for schools. At-risk students are dropping out of school, are denied grade level promotion, and are referred to special education programs in increasingly large numbers. School curricula are inevitably narrowed to focus on test taking, with an emphasis on drill-and-practice pedagogy and a more authoritarian relationship between teachers and students. Significantly, minority students experience these effects more frequently. Teachers are becoming more and more demoralized and placed in the role of technician rather than professional decision-maker. They are leaving the profession in record numbers.

These consequences have been the result of using single test measures for the purpose of classifying schools for rewards and punishments and determining student proficiency in school subject matter. This practice of using a single measure to make such high-stakes decisions about schools and students has been called invalid by most professional education associations, including the American Educational Research Association, the American Psychological Association, the National Council of Teachers of Mathematics, the National Council of Teachers of English, and many others. Increases in scores on such high-stakes tests do not necessarily mean increased achievement on the part of students, but often can be shown to be a form of grade inflation resulting from a greater emphasis on teaching test-taking skills associated with a specific test.

Further, the very idea that schools can be mandated to achieve a specified growth in test scores in a given period of time is not only arbitrary, but has deleterious effects on school improvement efforts. A recent study of the Philadelphia accountability system, which mandates an increase in test scores on its Performance Responsibility Index (PRI), describes the effects:
Intensive qualitative research in a sample of district schools identified schools that were pursuing effective strategies for improving instruction, but failed to reach their PRI targets. These schools were penalized even though they were on observable, defensible paths to improvement. The PRI underestimated the time needed to bring about meaningful change and created unrealistic expectations and burdens for staff. Rather than creating school cultures that encouraged organizational learning and internal capacity building, the PRI was a punitive measure that highlighted school failures. Focusing on preparation for the tests was perceived as the only sure way to avoid embarrassment and sanctions.

In sum, a flawed measurement system is being used to make inaccurate inferences about students and schools, causing great damage to public schools. Indeed, the stress on measurement has supplanted initial ideals and intentions about high standards for all students. A study of the decade-old reform accountability system in Delaware makes the following conclusion:
Over time, and for a variety of reasons, measurement-driven reform has taken the place of standards-based reform in Delaware. This overemphasis on testing has caused standards to take a backseat…Consequently, we believe that much of the distortion that we currently see in Delaware schools is attributable to this shift in focus.

What is needed is a better means for evaluating schools, an alternative to the present system of using high-stakes testing for school accountability. A new model, based on a different set of assumptions and understandings about school realities and approaches to power, is required. It must be centered on the needs of learners and on the intentions of having high standards for all rather than the prerequisites of a bureaucratic measurement system.

Assumptions of a Learner-Centered School Accountability System

In the realm of student learning, the question has often been asked about outcomes: what do we want students to know and be able to do as a result of schooling? From this starting place, school reform efforts have proceeded to address the thorny questions of how to attain these outcomes. Starting from desired outcomes is an important shift in how to think about what does or does not make sense in classroom instruction.

In the realm of school accountability, however, little attention has been paid to corresponding outcome-related questions. It has simply been assumed that schools should be accountable for improved student learning, as measured by external test scores. It has been largely assumed by policy makers that external tests do, in fact, adequately measure student learning. These and other assumptions about school accountability must be questioned in order to develop a more successful accountability model. It would be well to start from basic questions about the purposes and audiences of schools. For what, to whom, and by what means should schools be held accountable? Following are a set of assumptions about a new learner-centered school accountability system based on these questions.

For what should public schools be held accountable?
· Physical and emotional well-being of students
The caretending aspect of school is often take for granted, but is essential to quality education. Parents expect that their children are safe in schools and that adults in schools will tend to their affective as well as cognitive needs. Learning depends on a school climate that cultivates positive relationships.
· Learning
ü Student learning is complex and multi-faceted. It includes not only disciplinary subject matter, but also the thinking skills and dispositions needed in a modern democratic society. Student learning should be measured in multiple dimensions using multiple means, including assessments developed and administered at local and classroom levels.
ü Teacher learning is the most significant factor in student learning and should be fostered in multiple ways, compatible with the principles of adult learning. Schools must be provided with sufficient time and funding to enable teachers to improve their own performance, according to professional teaching standards.
· Equity and access
Given the history of inequity with respect to minority and underserved student populations, schools should be accountable for placing a renewed and special focus on improving equity and access, providing fair opportunities for all to learn to high standards.
· Improvement
Schools should be expected to function as a learning organization, continuously engaged in self-assessment and adjustment with respect to meeting the needs of their students. The capacity to do so must be ensured.

To whom should schools be held accountable?
Students, parents, and the local community are the primary clients for a school. Current accountability systems make the state and federal governments the locus of power and decision-making about school accountability. The primary clients should be empowered to make decisions about school accountability.

By what means should schools be held accountable?
· Multiple measures
Measures of school accountability should be multiple and include qualitative as well as quantitative approaches, taking into account local contexts, responsiveness to student and community needs, and professional practices and standards.
· Customized measures
Schools are complex and unique institutions that address multiple societal needs. A standardized approach towards school accountability cannot fit all.

Given these assumptions, what is the proper role of a government-developed and funded school accountability system? Here are some assumptions about the function of such a system:
· It should serve to improve student learning and school practices and to ensure equity and access, not to reward or punish schools;
· It should provide guidance and information for local decision-making, not classify or designate schools as successes or failures;
· It should reflect a democratic approach, including a balance of responsibility and power among different levels of government.

A Learner-Centered School Accountability Model

Models for an alternative accountability model have been proposed by various organizations and advocates. In particular, the National Center for Fair and Open Testing (FairTest) has been a leader in serving as a clearinghouse and advocate for improving assessment and accountability systems in the United States.

A framework for accountability currently employed in the business world called the Balanced Scorecard provides a useful perspective to consider for schools. This framework describes a four-part measurement system designed to give a comprehensive view of the state of the organization. The premise is that both outcomes and operations must be measured in order to have a feedback system that serves to improve the organization, not just monitor it. The four perspectives that form the framework for measurement are: (1) financial, (2) internal business, (3) customer, and (4) innovation and learning.

Applying this four-part approach to education, the following aspects of school performance provide the components of a learner-centered accountability model: (1) student learning, (2) opportunity to learn, (3) responsiveness to students, parents, and community, and (4) organizational capacity for improvement. Each of these aspects must be attended to and fostered by an accountability system that has a sufficiently high resolution to take into account the full complexity and scope of modern-day schools.

Student Learning

As many have pointed out, understanding what a student knows and is able to do is not a simple matter. Indeed, a primary source of the current problem with high-stakes testing is that merely administering a standardized test cannot make a fair and equitable assessment of student learning. Principles of high-quality assessment have been well-articulated by various organizations and should be followed. What is needed is a system that (a) is primarily intended to improve student learning; (b) aligns with local curricula; (c) emphasizes applied learning and thinking skills, not just declarative knowledge and basic skills; (d) embodies the principle of multiple measures, including a variety of formats such as writing, open response questions, and performance-based tasks (not just multiple choice); and (e) is accessible by diverse learning styles, intelligence profiles, exceptionalities, and cultural backgrounds.

Currently, there is a misfit between what cognitive science and brain research have shown about human learning and how schools and educational bureaucracies continue to measure learning. On the one hand, we now know that human intellectual abilities are malleable and that people learn through a social and cultural process of constructing knowledge and understandings in given contexts. And yet we continue to conduct schooling and assessment on the outdated paradigm that intelligence is fixed, that knowledge exists apart from culture and context, and that learning can be inculcated through the behaviorist model of stimulus-response. The idea that knowledge and understanding can be scientifically measured through an isolated and external instrumental approach ignores current knowledge about human learning. It fails to take into account the diversity and variability of intelligence. It also fails to take into account new understandings from the world of science itself:
[Society continues to have a] commitment to a rationalist style of thought that has influenced western societies for the past 400 years. When combined with the Newtonian mechanical school of physics which contended that we live in an orderly universe that is knowable through rational scientific methods, the Descartian dictum – I think therefore I am – has bequeathed us an intellectual paradigm which has led people to believe and act as if the world is knowable through logical, linear, rational strategies. This paradigm remains especially dominant in the hierarchies and bureaucracies of government who seem to respond to increasing complexity with ever more frantic efforts to impose control and certainty.
Since the early 1970s, proponents of the newer sciences such as quantum physics, molecular biology, Gestalt psychology, and ecology have challenged the conventional, rational paradigm. Its advocates have argued that rationality must be balanced by an ecological approach that looks at human and natural systems holistically.

As scientific measurement cannot truly “objectify” learning and rate it hierarchically, decisions about the quality and depth of an individual’s learning must be based on human judgment. To understand this is to understand that no test score can of itself trigger a decision about learning. While test scores and other assessment data are useful and necessary sources of information, a fair determination about a person’s learning can only be made by other people, most preferably by those who best know the person in his or her own context. This is why exit exams and other forms of high-stakes testing cannot be made equitable. A test alone cannot tell the whole story and therefore cannot carry the burden of a high-stakes decision. A reasonable process for determining the measure of student learning could involve local panels of teachers, parents, and community members who review data about student performance and make decisions about promotion, graduation, placement, and so on.

What is missing in most current accountability systems is not just a human adjudication system, but also a local assessment component that addresses local curricula, contexts, and cultures. A large-scale external test is not sufficient to determine a student’s achievement. District, school, and classroom assessments must also be developed as part of a comprehensive means of collecting data on student learning. The states of Maine and Nebraska are notably developing just such systems. While the promise of higher resolution data about student learning gives this work impetus and importance, several factors have emerged as foundational for its successful implementation.

Most importantly, locally developed assessments depend upon the knowledge and “assessment literacy” of teachers. Most teachers have not been adequately trained in assessment and need substantial and ongoing professional development to develop valid and reliable tasks and effective classroom assessment repertoires. This means that an investment must be made in teacher learning about assessment. The value of such an investment is not only in the promise of improved classroom instruction and measurement. Research also shows that improved classroom assessment results in improved student achievement on external tests.

Teachers and administrators will also need the time, resources, and guidance to do the development work that is needed to get such a local system off the ground. While the psychometric standards of validity and reliability appropriate to high-stakes standardized multiple choice tests are not the same as those needed to ensure high quality assessment in a more performance-based, low-stakes environment, there are still important criteria to meet in a locally developed system. Moreover, the decisions made by local panels must be firmly grounded in data using transparent and equitable processes. An important role must be played by state departments of education in providing training for assessment development and decision-making and in auditing the ongoing quality of local assessment implementation.

Lastly, the need to determine the effectiveness of the larger state school system can either support or undermine such local efforts. If state or federal agencies approach this need by way of requiring aggregated data from local to state levels, local decision-making is necessarily weakened and an undue emphasis is placed on standardized methods. If, however, the state and federal agencies do not rely on local assessment systems to gauge the health of the larger system, much may be gained. In New Zealand, for example, a system of educational monitoring is in place that entails using matrix sampling (like NAEP) on tasks that include one-to-one videotaped interviews, team tasks, and independent tasks. No stakes are entailed for schools or students. The data is profiled and shared with schools for the purpose of teacher professional development and as a means of developing model tasks for local assessments. Such a system supports rather than undermines local assessment efforts.

Schools and districts should be accountable not only for student outcomes, but also for the development of a local assessment system and for fair and accurate decisions made about students. The results of student assessments are only part of the larger accountability model.

Opportunity to Learn

How can students be expected to meet high standards if they are not given a fair opportunity to learn? This is the yet unanswered question with respect to school accountability. Schools should be accountable to provide equitable opportunities for all students to learn and we must develop ways to determine how well they do so.

At the heart of the matter is the responsibility for this must be shared by the district and state. The inequitable funding of public schools, particularly the disparity between the schools of the haves and the have-nots, places the schools of disadvantaged students in unjust and often horrifying circumstances. Over the past decade, there have been lawsuits in various states attempting to redress this imbalance, largely a factor of dependence on property taxes for school funding. Yet not a great deal of progress has been made.

An interesting lawsuit was initiated against New York State in 1995 by a group of parents from New York City public schools, represented by the Campaign for Fiscal Equity (CFE v. State of New York). A lower court found that the inequity in funding was unconstitutional ($13,000 per student in many districts compared to $9,623 in the city, which educates over 70% of the state’s economically disadvantaged students, over 80% of its limited-English-proficient students, and 51% of its students with severe disabilities). Then the appellate court, on an appeal filed by the governor of the state, overturned the decision, basing its decision on the finding that the state is only required to provide the equivalent of an eighth to ninth grade education to all its students. In June, 2003, the Court of Appeals, New York’s highest court, reversed the finding of the appellate court and affirmed the lower court finding, requiring the legislature to remedy the inequity within a year.

A study convened New York City high school students to research and discuss the case prior to the most recent reversal by the Court of Appeals. Among its data and findings is the following excerpt:

“If all schools have to give is an eighth- or ninth- grade education, why are they making us take the Regents?” one student asked, as the rest murmured their assent. These students recognize that they live at the heart of a policy paradox: a raising of standards required for a high school diploma, along with a declaration that the state has no responsibility to educate students to the levels required for a high school diploma. While this paradox may escape both politicians and policy-makers, it is felt deeply by the students upon whose heads it comes to rest.

Clearly, inequitable funding is a central problem in providing opportunities to learn for all students. But there is more to this issue than just funding. How should we define and operationalize the opportunity-to-learn construct? How will we measure it? How can an accountability system foster it?

At a minimum, one might expect that schools and school systems provide qualified teachers, adequate instructional materials, and sound facilities. This is the contention in a recent lawsuit, Williams v. State of California. A synthesis of expert testimony in that case reports the following findings:

· Qualified teachers, relevant instructional materials that students may use in school and at home, and clean, safe, and educationally appropriate facilities are fundamentally important to students’ education. They enable students to learn the knowledge and skills that the state has specified as important. They promote students’ chances to compete for good jobs and economic security. They provide students with the tools to engage in civic life as adults. The consequences of not having access to such teachers, materials, and facilities are particularly harsh in California’s current high-stakes, standards-based education system.

· Many California students do not have the teachers, materials, and facilities that are fundamental to their learning and that are enjoyed by the majority of California students. The burdens of these serious shortfalls are borne most heavily in high-poverty schools, disproportionately attended by children of color and students still learning English. Such students are often housed in overcrowded, deteriorating facilities. Their schools frequently lack critical instructional resources. Often these are also schools with the fewest qualified teachers and the schools in which student achievement and college-going rates remain very low. The insufficient supply and poor quality of these educational basics create significant obstacles for students as they attempt to meet the content standards the State has set, pass State tests that are required for grade-to-grade promotion and high school graduation, and qualify for competitive opportunities in college and the workforce. In some schools, these poor conditions breed social alienation and pose real threats to students’ health and well-being.

The report places the responsibility for providing such opportunities to learn squarely on the state and argues for a modification of the state’s governance of the educational system in the following ways:

· Set state standards that specify the resources and conditions that are essential for teaching and learning;

· Base the school funding system on what providing essential resources and conditions actually costs, with adjustments for cost differences in schools serving different communities and students;

· Expand the state accountability system to accurately and fairly measure and report learning resources and condition, as well as achievement test scores [the model in this paper argues for a much broader view of assessment than “achievement test scores”];

· Establish unambiguous lines of state, regional, and district responsibility for ensuring that all students have these learning resources and conditions, and develop mechanisms that hold the appropriate officials at each of these levels accountable;

· Ensure that the accountability system is reciprocal – i.e., that it includes a two-way flow of accountability information and provides legitimate roles for local communities, parents, and students in holding the system accountable.

In a seminal work published in 1989, Oakes describes a more elaborate framework for opportunity to learn. Here the emphasis is placed on the conditions that support or detract from high quality classroom teaching and learning. Oakes details indicators for three contextual dimensions of schools: access to knowledge, professional teaching conditions, and press for achievement:

· Access to knowledge: teachers’ qualifications and experience; instructional time; course offerings; class grouping practices; materials, laboratories, computers, and equipment; academic support programs; academic enrichment activities; parents’ involvement in instruction at home or at school; opportunities for staff development; teachers’ beliefs about the importance of challenging academic study for all students.

· Professional teaching conditions: teachers’ salaries; pupil load/class size; teachers’ time for professional non-teaching work; teachers’ time for school-based goal setting, staff development, program planning, curriculum development, instructional improvement, collaborative research; teachers’ involvement in decision making; teachers’ certainty about their ability to influence and achieve school goals; teachers’ autonomy/flexibility in implementing curriculum and instruction; administrative support for innovation; clerical support for teachers’ non-instructional tasks.

· Press for achievement: focus on academics; graduation requirements; graduation rates; students’ participation in challenging academic work; school-wide recognition of academic accomplishments; teachers’ expectations about student’s ability to learn; uninterrupted time for class instruction; administrative advocacy and support for challenging curriculum and instruction; quantity and type of homework; the extent to which teaching and learning are central to teacher evaluation.

Darling-Hammond considers these indicators within the perspective that schools should be accountable for “fair and humane treatment” of students and folds them into a set of standards for professional practice. Among these standards are the following, drawn from the principles of the Coalition of Essential Schools):

· Students should be treated with respect and dignity in an environment that stresses trust, fairness, and a climate of unanxious expectation;

· All students should be well known, as learners and as individuals, to those who have responsibility for their development. Teaching and learning should be personalized to the maximum extent feasible.

The current interest in and development of small schools, especially at the secondary level, is based upon such standards. Research findings and experience suggest that such a personalized learner-centered approach lead to many improved outcomes for students, including improved academic achievement.

As such standards for professional practice are articulated, the accountability question arises as to how to monitor and report on them. Clearly this cannot be done through the proxy of testing. What is needed is a means of observation in schools and classrooms in order to determine the degree of adherence to these standards. Two aspects of this must be considered: the quality of individual teachers and the quality of the school as a whole.

Teacher evaluation has received a great deal of criticism for being ineffective. The hit-and-run observations so often done by principals do little to determine whether teachers are meeting established professional teaching standards. Unions have been described as more interested in protecting its membership than ensuring high quality teaching. A promising development that has potential for breaking through this impasse is the recent initiation of peer review processes by a number of teacher unions. Adam Urbanski, president of the Rochester Teachers Association and director of the Teacher Union Reform Network (TURN) has been a leader in advocating for and implementing such teacher evaluation processes. In a recent manuscript, he describes how the process should work:

There is a need for meaningful summative evaluations of teachers at key junctures: tenure and movement between career levels. There must also be useful formative evaluations in between. Evaluation methods should include the following:

· Some classroom observation by peers and supervisors structured by a narrative instrument (not checklist) based on professional standards such as those of the National Board for Professional Teaching Standards (NBPTS) and framed by the teacher’s goals for the lesson/unit;

· Information from previous evaluations and feedback, such as structured references from colleagues and other supervisors;

· Portfolios that might include examples of teaching syllabi, assignments given, feedback given to students and samples of student work, feedback received from parents and students as well as colleagues, data on student progress, teaching exhibitions such as videotaped teaching samples, professional development initiatives taken, and structured self-evaluation.

All summative evaluation decisions about promotions or continued employment should be made by a specially established committee of teachers and administrators.

Urbanski goes on to describe safeguards for due process and for preventing malpractice. He also describes how such a process could be used in conjunction with professional development for improving teaching and school practice.

In a recent essay, Lee Shulman, president of the Carnegie Foundation, remarks on the ethical dimension to teaching, saying that “a professional actively takes responsibility; she does not wait to be held accountable.” Most teachers consider themselves professionals and hold themselves accountable as individuals to their students and parents. It is an apt role of professional teacher organizations to extend this accountability to include collective expectations that are visible to the public and subject to a review system that uses acknowledged standards for teaching to support, maintain, and enforce that professionalism.

Various review processes have been implemented in this country and others in order to evaluate the performance of a school as a whole with respect to opportunities to learn and professional practices. Variations of inspectorates and school quality reviews have been developed in New York, Rhode Island, Maine, and other states, as well as in Britain, New Zealand, Australia and other countries. School accreditation processes have now adopted similar standards-based review processes, rather than relying upon the outdated check for “inputs.” A possible model not yet adapted for a school accountability process is the benchmarking process used in business. Surveys and questionnaires have often been used to collect data on school and classroom practices.

Responsiveness to Students, Parents, and Community

It has become almost a set of platitudes in the lexicon of school reform that teachers should be attentive to the needs of students, schools should invite and cultivate parent involvement, and districts and schools should include community interest groups in discussions about priorities and policies. And yet, there is no sign that any of these stakeholders in the system feel more empowered by the new approach to standards and accountability. Indeed, the current accountability system that focuses strictly on testing and prescribed outcomes moves power and decision making away from the primary clients of the educational system and more and more into state and federal agencies. As high-stakes testing dictates the curriculum, less and less choice is available for students. Parent or community concerns about what is happening in the classroom and to their children are less important to schools than meeting state mandates.

Schools should be accountable to their primary clients. It must be the role of an accountability system to set the conditions for this and to guide appropriate implementation efforts. This means that the role of state and federal agencies must change from one of command and control to one of support and feedback. The increasing centralization of power that has come with high-stakes testing must give way to a more decentralized system, where educational ends as well as means are determined locally, guided by standards for student learning and opportunity to learn, as outlined above. One example of such decentralization can be seen in Maine and Rhode Island, where the state has determined that state testing can count for no more than 10% of the weight in the local assessment system for determining student graduation.

Schools should be accountable for their responsiveness to students. A longstanding issue in education has been the lack of attention to student needs, whether that be individual learning styles, exceptionalities, group cultural norms and expectations, gender differences, or language barriers. It is well known that differentiation is needed in curriculum and instruction and that standardization creates inequities. In the world of special education, the individualized educational plan, which targets learning goals as well as means, is well accepted. In multicultural education literature and practice, the use of cultural context is deemed essential to engage students. Teacher standards, such as those articulated by the National Board for Professional Teaching Standards, highlight the need to respond to the needs of the specific students in a given teacher’s classroom. Responsiveness to students means that curricula must be adapted, instructional time frames must be adjusted, student choices must be given. A prescribed curriculum for all, enforced by exit exams, gateway tests, and so-called grade-level expectations prevents this. What is needed is that each student is presented with important and challenging studies, not that one “best way” is set in place for all. This is why local decision-making about curriculum is critical.

One of the premises in the present definition of standards-based reform is that it is appropriate for states to set the learning goals and outcomes in place, monitor student achievement of these outcomes with a testing system, and then apply consequences based on the success or failure of students in this test. The understanding is that local districts and schools would be free to determine what curriculum to use in order to achieve these outcomes. Supposedly, schools and districts are given more freedom in this system than in an input-based control model that focuses on adequacy of materials, time-on-task, and other resource or methodology issues. In fact, as teachers well know, tests are based on content and assume a given curriculum. A high-stakes achievement test developed and administered at the state level must be based on a state curriculum framework that is very well specified, as a matter of validity. This inevitably preempts local curriculum choices.

Moreover, testing format and content often lead to a particular form of pedagogy. For example, a basic-skills, multiple-choice test leads to more drill-and-practice in the classroom whereas a more performance-oriented assessment that focuses on thinking skills leads to more writing and discussion. These effects are understood in the psychometric community and included within the parameters of consequential validity for a given test. It is unlikely that a low level multiple-choice test or even short-answer test is valid for the purpose of assessing learning standards that include the application of knowledge or complex thinking skills. Thus, a state test also strongly influences classroom pedagogy.

The concept that the present accountability system provides greater local flexibility and decision-making about curriculum and pedagogy is an illusion. Students do not see high-stakes testing as a means of improving their learning experience. A learner-centered accountability system requires that decisions on curriculum and instruction be made locally.

Saying this, however, obliges us to acknowledge that one of the existing problems meant to be addressed by the present accountability system is the history of school unresponsiveness to students. The achievement gaps we are facing are painful evidence to many parents, community organizations, and educational policy makers that schools are not working when left to their own decision-making about curriculum and instruction. If high stakes tests don’t improve this, what will work better?

The answer must be to look directly to parents and communities as the primary stakeholders for holding schools accountable, not to state and federal agencies as their proxies. There are many examples of local community organizations, especially in urban areas, taking on the mantle of responsibility for insisting that schools are responsive to the needs of their children.

Across the country, parents and community members in distressed, low-income neighborhoods are taking action to improve their local schools. They examine school performance data that show dismal student outcomes, and they raise demands to secure greater accountability as well as sufficient resources to ensure that their schools can succeed. They get involved in school board elections, form coalitions, work with the media, and engage a broad spectrum of public actors to improve public education. All these efforts are making clear, strong impacts. They have won a new small schools policy in Oakland, California; directed resources towards building new schools in the poorest, most underserved communities in Los Angeles; and brought new after-school programs to Washington, DC, schools. They have forced the removal of ineffective principals and superintendents in New York City and Mississippi, worked with teachers to improve student achievement in Chicago, and brought more rigorous math and reading programs to schools in Philadelphia.

As an example, the Oakland Community Organization (OCO), led by the leaders of eight local churches, motivated by severely overcrowded and underperforming schools, worked with the Oakland Unified School District, developed a plan for opening ten new small schools in three years. New schools were developed and opened and the OCO took on the role of supporting and monitoring them, working with parents and teachers. A group of parents at one school, in cooperation with a doctoral student from UC Berkeley formed an action research committee that led to greater parent-teacher communication, a parent center within the school, and various forms of community outreach.

In a recent interview, Steve Jubb, director of the Bay Area Coalition for Equitable Schools (BayCES), who works in collaboration with the OCO, discussed the perspective necessary for such community organizations to be effective. Jubb describes the need for relational accountability, where parents and community organizations become a force to change the “paternalistic” culture of schools and where school officials and families are accountable to each other.
The real power for positive change lies in the relationships between the primary actors, not in the infrastructure. Infrastructure should serve to strengthen our relationships, not prevent us from having them. You have to change the power relationships to do this-to get the level of partnership required for equitable and high quality systems of schools serving low income families.

Norm Fruchter, director of the New York University Institute for Education and Social Policy, writes about developing community involvement as an accountability model. He describes the theory of action for “new power collaborations:”

These collaborations engage non-schooling sources of expertise to initiate and drive schooling reform, and organize community will to mobilize and sustain the political force necessary to implement reform. Thus, the theory of action of these new power collaborations first posits that school systems, acting by themselves, can muster neither the educational expertise not the political will necessary to carry out or sustain significant reform. Next the theory of action assumes that external intermediaries are required to mobilize the necessary educational expertise and political will, through community engagement, community organizing, or both. Finally, the theory of action posits that the changes produced by these processes will be sequential and cumulative.

Using responsiveness to students, parents, and community as a form of accountability must go beyond PTA-like support efforts or many existing efforts to gain parent involvement, which are often focused on gaining parent support for existing school practices. It must also go beyond common efforts at gathering survey information about satisfaction. Real accountability to the primary clients for schools entails shifting power relationships.

What vehicles might be used to elicit and enact new relationships? One existing effort is the Teaching to Change LA virtual school report card system. On this website, students and parents are invited to report on the conditions in their schools in a way that goes beyond existing state-sponsored school report cards, featuring issues such as the quality of instructional materials or efforts to promote a safe and democratic school environment. In this way, information about the schools is made public from a school’s clients in a way that is not controlled by school authorities. Another example is the home visitation program in Sacramento worked out through the Pacific Institute for Community Organizing, in which teachers systematically visit the homes of their students asking parents for their insights and expectations. Still another is the Industrial Areas Foundation (IAF) in Texas:
Using community organizing strategies to build strong parent and neighborhood constituency groups in low-income school neighborhoods, the Texas IAF work has forged links with principals and teachers to strengthen literacy curricula, improve student reading and math performance on state tests, secure district and state funding for after-school programs, and mobilize a powerful set of constituencies for education reform.

In order to integrate such a responsiveness component into a school accountability system, it will be important to create local school-based councils that have real power to effect school change. Fruchter recommends that a convening organization play a central role:
…the convening organization should be an established group with strong leadership, a reputation for integrity and effectiveness, political savvy, and commitment to community improvement. Ideally, the convening organization would also have a well-developed relationship with the school system, and access to and familiarity with both public and private funding streams.
…our ideal collaborative…would include representatives of the following sectors:
· The school system
· The teachers union
· City and county departments and agencies
· Youth development organizations
· Children, youth, and family service providers
· Corporate and business groups
· Community groups and organizations
· Faith-based institutions
· Civic groups
· Advocacy organizations
· Local and state elected officials
· State agencies

Schlechty builds a case that district-level “guiding coalitions” should be chartered as an antidote to both the current distant and bureaucratic approach of state accountability systems and the lack of moral leadership from school boards which are all too often factionalized and focused on short-term interests:

Community leaders who are concerned about the futures of their communities and their schools should join together to create a non-profit corporation intended to support efforts of school leaders to focus on the future and to ensure that lasting values as well as immediate interests are included in the education decision-making process. It would also be the function of this group to establish a small sub-group of the community’s most trusted leaders who would annually evaluate the performance of the school board as stewards of the common good and would make these evaluations known to the community…
In a sense, the relationship between the school district and the monitoring function of the new corporation should be something akin to the relationship between the quality assurance division of a corporation and the operating units in the corporation…
When the data indicate that goals are not being met, the president of the corporation, working with the superintendent and the board of education, would seek to discover why this was the case, and would seek as well to create new approaches that might enhance the prospect of achieving the stated goals and the intended ends. It is not intended that the new corporation simply identify problems and weaknesses, it is intended that the leaders of this organization also participate in the creation of solutions and participate in creating support for solutions once they have been identified or created.

These councils could review accountability information from state and local assessments as well as from school quality review processes and make recommendations to school boards about school policies and priorities. They could be responsible for overseeing the development and implementation of school improvement plans. Their relationships with local school administrations, school boards, and state departments of education would need to be explicated by law so as to ensure a system of shared responsibilities and empowerment. Questions about how to sustain such councils and ensure that they do not pursue narrow agendas must be determined. How councils are composed in urban settings will likely vary and be different than those in rural or suburban settings. Standards and acceptable variations for councils will be important topics for public discussion.

Organizational Capacity

If schools are going to be held accountable to high levels of performance, the question arises: do schools have the internal capacity to rise to those levels? This question has been addressed for many years, and within the last decade has been specifically related to the issue of school reform. Newmann, King and Rigdon define the issue this way:
Proposed ingredients of organizational capacity include teachers’ professional knowledge and skills, effective leadership, availability of technical and financial resources, and organizational autonomy to act according to the demands of the local context…This formulation makes sense, but, in our view, needs to go one step further. The critical defining feature of organizational capacity is the degree to which the human, technical, and social resources of an organization are organized into an effective collective enterprise…For schools the key to effective coordination and organization of these resources is shared commitment and collaboration among staff to achieve a clear purpose for student learning.

The issue of meaningful and ongoing teacher professional development is especially salient to whether or not schools is capable of enabling all students to meet higher standards of performance. While it is generally agreed that the quality of the teacher is at the heart of student learning, it is too often the case that time, resources, and high-quality professional development opportunities have been lacking. In many cases, teachers have been subjected to a training model that asks for faithful implementation of an innovation rather than an inquiry into how an innovation might be adapted to meet the needs of the specific context. More and more, teachers are asked to use allotted professional development time to do the work mandated by new accountability systems, rather than to develop their own professional knowledge and skills.

Little suggests that, in a climate of educational reform, the nature of teacher professional development must itself be reformed. She states:
…the most promising forms of professional development engage teachers in the pursuit of genuine questions, problems and curiosities, over time, in ways that leave a mark on perspectives, policy, and practice. They communicate a view of teachers not only as classroom experts, but also as productive and responsible members of a broader professional community and as persons embarked on a career that may span 30 years or more.

Little offers six principles for high quality professional development:
1. It offers meaningful intellectual, social, and emotional engagement with ideas, with materials, and with colleagues, both in and out of teaching;
2. It takes explicit account of the contexts of teaching and the experience of teachers;
3. It offers support for informed dissent;
4. It places classroom practice in the larger contexts of school practice and the educational careers of students;
5. It prepares teachers (as well as students and their parents) to employ the techniques and perspectives of inquiry;
6. It is governed in a way that ensures bureaucratic restraint and a balance between the interests of individuals and the interests of institutions.

In studying secondary schools, McLaughlin and Talbert found that in order for school reform efforts to take hold, meaningful professional development must be enacted in the context of strong professional communities. They point out that such communities must be focused on a shared sense of teacher inquiry:
Opportunities for teachers to talk with colleagues about teaching, consider new ways of doing things, and hammer out shared understandings about goals were common across diverse environments where practices were rethought in ways that benefited both teachers and students. Teachers’ ability to respond effectively to the diverse needs, interests, and talents students bring to their classrooms and to implement principles of teaching that motivate reform will depend upon their ability to have these relationships.
… [But] not all strong teacher communities develop knowledge to improve their collective success with students. Policy to support such communities must go beyond providing opportunities for social interaction to engage cultures of practice. The challenge for policy and practice posed by the vision of teacher learning communities is to engender norms of inquiry, innovation, and shared accountability to support effective teaching practice and rewarding careers.

Schools must also attend to the issue of teacher empowerment. Ingersoll documents how teachers are controlled and disempowered in various ways, especially in matters related to the social and behavioral issues of the schools. As this control increases, it leads to a decreasing sense of efficacy and professionalism and an increasing sense of job dissatisfaction. This becomes a factor in the drain from the profession behind the growing teaching shortage:
The teacher who has little control and power is the teacher who is less able to get things done and thus the teacher with less credibility. Students can more easily ignore such a teacher – indeed a lack of influence and credibility invites challenge. Principals can more easily neglect them. Peers may be more likely to shun them. In such cases teachers may feel that they have little choice but to turn to manipulative or authoritarian methods (that is, to adopt the “bureaucratic personality”) to get the job done, which may simply exacerbate tensions with students and fellow staff. This, in turn, would likely lead such teachers to feel less commitment to their teaching job or to the teaching career.

Data from Ingersoll’s study has been used to demonstrate that the current teacher shortage is not a result of teacher recruitment, but rather one of retention. As curricular and instructional decisions are increasingly dictated by state and federal mandates, this problem is likely to worsen, creating a systemic problem beyond a given school’s organizational capacity.

In order to support and sustain teachers and school improvement efforts, Hargreaves suggests that school leaders and policymakers must take an ecological perspective:
Promoters of sustainability cultivate and recreate an educational environment or ecosystem that possesses the capacity to stimulate ongoing improvement on a broad front. They enable people to adapt to and prosper in their increasingly complex environment. Rational, standardized scientific efficiency is the enemy of healthy and creative diversity. It produces overly simple systems that are too specialized to allow the learning and cross-fertilization that is necessary for healthy development.

Hargreaves emphasizes that an important role for principals is to distribute leadership. Rather than take a type of heroic stance in leading schools into reform, it is important for principals to share power with teachers and others. This not only benefits teacher efficacy, but works towards the institutionalization of change. It is a means of developing lasting organizational capacity.

To be an effective collective enterprise, a school must develop an internal accountability system. That is, it must take responsibility for developing goals and priorities based on the ongoing collection and analysis of data, monitor its performance, and report its findings and actions to its stakeholders. Many schools have not moved past a condition where individual teacher responsibility rather than collective responsibility is the norm. In these schools, it is not likely that an external accountability system will have a positive effect on teaching and learning.

On the other hand, there are some schools that have developed an internal and collective accountability system. In a five-year study of twenty-four schools committed to school reform, Newmann et al found that organizational capacity varied significantly among these schools and was not related to accountability processes. Further, they found that those schools existing within strong external accountability systems tended to have low organizational capacity. This resulted primarily from the discord generated when external accountability systems contradicted existing internal accountability systems. They conclude:

We showed that when highly specific prescriptive standards connected to high-stakes consequences are mandated by external authorities, this can deny school staff both the “ownership” or commitment and the authority it needs to work collaboratively to achieve a clear purpose for student learning.

What is the proper role of district, state and federal agencies in fostering the kind of internal accountability mechanisms that would lead towards improved organizational capacity? Newmann et al address this issue in their conclusions:

Districts and states could support internal accountability in several ways…they can set expectations that individual schools establish their own standards for performance and a responsible reporting system. They can support staff development opportunities for teachers within a school to formulate performance goals and ways to implement them. Districts and states can also establish and reinforce support networks of reform-minded schools to assist in sharing standards, assessment techniques, and review procedures for evaluation of student learning and school goals.
The point of such activities is not simply to craft procedures that “let schools do their own thing.” It is to elevate school performance by improving the standards to which schools aspire. External agents can make important substantive contributions by offering concrete examples of high standards for student performance in specific curriculum areas, approaches to assessment that demand high performance, and reliable ways of evaluating student performance on the assessments. In these ways, districts and states, working with school networks and professional organizations, can, without imposing uniform tasks or tests in all schools, offer leadership in the creation and definition of high standards for student achievement. And through such strategies they are more likely to enhance school organizational capacity.

How can organizational capacity be included as an element of school accountability? Can we hold schools accountable for having such capacity? On the one hand, it does seem fair to expect schools to provide meaningful professional development opportunities for teachers, to empower teachers to make decisions, to provide sustaining leadership, to develop internal accountability mechanisms. But schools also depend on districts and states to provide the time, resources, and personnel needed to live up to those responsibilities. As with opportunity to learn, this must be a shared area of accountability, across levels of jurisdiction.

The New Role of the State

For a learner-centered model of school accountability such as the one outlined here to succeed, there must be a system where states and districts are jointly responsible with schools and communities for student learning. Reciprocal accountability is needed, where one level of the system is responsible to the others and all are responsible to the public.

The role of state and federal agencies with respect to school accountability is much in need of redefinition. Agencies at these levels should not be primarily in an enforcement role. Rather, their role should be to establish standards for local accountability systems, to provide resources and guidance, and to set in place processes for quality review of such systems. Certainly there should be no high-stakes testing from the state and federal levels, no mandatory curricula, and no manipulation through funding. Where there are clear cases of faulty local accountability systems – a lack of appropriate local assessment systems, adequate opportunities to learn, responsiveness to students, parents, and community, or organizational capacity – supportive efforts should be implemented.

How will states know which schools need help if there is no number to go by, no accountability index? There will need to be an agency for accountability review. The agency should develop and facilitate a school quality review of local accountability systems based upon expressed standards for such systems.

In order for such reviews to serve the purpose of school improvement, it is essential that the data collection be done through a combination of school self-assessment and collegial visitations done in a “critical friend” manner. Findings from such a process should not be stated or used in a bureaucratic and judgmental way, but rather should be given to a local oversight committee charged with evaluating school accountability. As with all aspects of a learner-centered accountability system, the quality and effectiveness of a review system depends upon the time, resources, and institutional support given to it.

Are there any circumstances in which a state should intervene forcibly in a school or district? If an accountability system is to work towards school improvement for all schools, does it not need such “teeth?” This question must be addressed in a way that acknowledges the multi-level nature of this school accountability model. One might envision at least three cases where the state would take on a more assertive role: (1) to investigate claims or appeals from students, parents, or the local community that the local accountability system is not meeting the standards set for such systems; (2) to require local schools and districts to respond to findings in the data that show significant student learning deficiencies, inequity in the opportunities to learn for all students, or lack of responsiveness to students, parents, or communities; (3) to provide additional resources and guidance to improve the organizational capacity of the local school or district. Is it conceivable that a state might take over a local school or district in this model? Yes, but only after the most comprehensive evaluation of the local accountability system has shown that there is no alternative and only on a temporary basis.

It is of great importance to the health of our public schools that we begin as soon as possible to define a new model for school accountability. Schools can and should be held accountable, but to continue the current model of using high-stakes testing is a recipe for public school failure. We may indeed be a nation at risk.