The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.
Volume II, Number 1, Winter 1996
Issue Topic: Results-Based Accountability
Questions & Answers
Education Outcome-Based Accountability Systems: Challenges and Lessons
Dr. Susan Furhman is the Dean of the Graduate School of Education at the University of Pennsylvania. An expert in education reform, Dr. Furhman has studied state efforts to develop and implement education outcome-based accountability systems. We asked her to describe some of the challenges and lessons learned in the implementation and development of these systems.
What do you see as the primary political challenges states face in developing and implementing education outcome-based accountability systems?
Decisions about accountability are intensely political. In many ways, accountability policies are the state's clearest expressions of what is expected of schools and districts in return for the delegation authority to provide schooling and state aid. They carry within them policymakers' understandings of what state constitutional responsibility for education means—what schools must deliver, and what the state must guarantee they deliver—to students. In some states, the accountability policies bear the entire burden of state-local relations because state leaders assume districts and schools will not pay attention to anything except what is required and monitored. Accountability policies certainly affect the focus of school personnel; they turn their attention to the indicators used by the state. Therefore, decisions about those indicators, and the consequences related to their achievement, are profoundly important to local educators, to key stakeholders and their associations, and to policymakers.
Some of the issues surrounding the design and implementation of performance-based systems are recurrent themes in state education politics: constituency pressures moderate policymakers' initial stances, resulting in efforts that provide more local leeway than originally planned; resource constraints affect policy design and implementation; and the instability of the political environment threatens the stability of policy and its translation into practice. Some political issues are stimulated by the changes in accountability themselves: new approaches are poorly understood by and even opposed by elements of the public, raising the possibility of backlash; and the performance focus is difficult to maintain in the face of counter pressures to hold schools and districts accountable for input and process standards. Policymakers must work hard to explain the new systems to the public.
What are the technical obstacles states face in developing education accountability systems?
One technical challenge is designing appropriate and adequate measures, particularly as testing technology is changing and newer tests raise reliability and validity concerns. For example, open-ended responses and portfolios tend to be scored differently by different people. States hoping to use such measures because they promote more ambitious instruction and because they tap students ability to think critically and communicate better must spend considerable resources on training scorers to improve reliability. Another issue is deciding what criteria to establish for success, such as whether to use absolute criteria and/or criteria based on progress. When developing accountability systems linked to emerging standards for student learning in key content areas, policymakers want to establish levels that indicate various degrees of mastery of the subjects. But shouldn't they also recognize and perhaps reward schools progress toward such mastery?
Further ReadingCohen, D. K. (in press). Rewarding teachers for students' performance. In S. Fuhrman & J. O'Day (Eds.), Rewards and reform: Creating educational incentives that work. San Francisco: Jossey-Bass. Clotfelter, C. T., & Ladd, H. F. (in press). Picking winners: Recognition and reward programs for public schools. In H. Ladd (Ed.), Holding schools accountable: Performance-based reform in education. Washington, DC: Brookings Institution. Fuhrman, S. H., Abelmann, C., & Elmore, R. F. (in press). The new accountability in state education policy. In Performance-based strategies for improving school. Washington, DC: Brookings Institution. Hambleton, R. K., et al. (1995). Review of the measurement quality of the Kentucky Instructional Results Information System 1991-1994 . Frankfort, KY: Kentucky General Assembly, Office of Education Accountability. |
A third challenge lies in adjusting for student background in a way that is fair for both students and schools. Not controlling for differences in socioeconomic characteristics permits unfair comparisons, but controlling for them may send the message that more disadvantaged students are subject to lower expectations. One way out of this dilemma is to hold schools responsible only for the value they add to student learning, no matter what the composition of their student bodies. However, holding schools accountable only for “value added” to student learning may require frequent, costly testing because it would be necessary to have scores on each individual and credit a school for score gains made only by those students that remained in that school during the period measured. The higher the mobility among schools, the more important and challenging tracking individual students becomes. Generally, the newer the testing technology, the higher the cost, so it is difficult to move to more authentic testing while trying to track each student's scores in a way that is fair to schools.
What are some innovative ways states have obtained the financial resources for the development and implementation of outcome-based accountability systems?
Performance-based systems require technical expertise in assessment and evaluation, and sufficient capacity to help low performing schools and districts improve. States tend to underinvest in both these areas, because the political imperative is to get money out to districts, not to keep it in the capital supporting “bureaucrats.” As competition for state dollars gets fiercer, especially as federal budget cuts come on line, there will be even more pressure to keep costs down. There's a certain irony here because many cost-cutting politicians are the first to argue for instituting performance-based policies. States have found some novel ways, however, to support the development costs. For example, they join collaboratives, such as the New Standards Project, with other states. As to assistance, states and districts are increasingly relying on reform networks, such as Accelerated Schools and Success for All, or subject-matter networks of teachers and on universities to help low-achieving schools.