The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.
Volume ll, Number 3, 1996
Issue Topic: Results-Based Accountability
Evaluations to Watch
The System Accountability Project for Children's Mental Health: Understanding Outcome Information in Context
Mario Hernandez and Sharon Hodges from the System Accountability Project for Children's Mental Health describe the project's goal to better understand the impact that utilizing measurable outcomes has on the planning and delivery of services.
Child-serving agencies produce outcomes for children and families every day—regardless of whether these outcomes are accurately tracked, publicly articulated, or even meet the stated goals of those agencies. Despite increasing demands that child-serving agencies be accountable for achieving measurable outcomes, public agencies continue to expend enormous energy in collecting information that has little or no bearing on what they are accomplishing.
Researchers with the System Accountability Project for Children's Mental Health at the Florida Mental Health Institute (FMHI), University of South Florida are investigating how information about outcomes affect service delivery, how outcome information affects providers associated with the larger systems under study, and how outcome information impacts the involvement of families and other key stakeholders in service system planning. The goal of the System Accountability Project is to better understand the impact that utilizing measurable outcomes (such as school attendance, juvenile justice recidivism, and out-of-home placements) has on the planning and delivery of services.
The initial focus of the System Accountability Project is the System Accountability Study, a five-year investigation of the best practices and challenges of child-serving systems which have established approaches for tracking outcome information being conducted for the Research and Training Center for Children's Mental Health at FMHI. Three exemplary sites have been selected to participate in this phase of the System Accountability Study: the Texas Children's Mental Health Plan (a statewide effort), Pressley Ridge Schools (headquartered in Pittsburgh), and Santa Cruz County (California) Children's Mental Health Services.
This research involves interviews, site visits, and documents reviews at each of the participating sites. The investigation of the best practices of these systems is based on a framework for building outcome accountability in children's mental health called the Ecology of Outcomes. The Ecology of Outcomes framework provides a foundation for the utilization of information with the goal that outcome information can provide opportunities for learning and self-correction in decisions regarding service planning and delivery.
Operationalizing Outcome Accountability
The Ecology of Outcomes is a framework, a way of considering what an outcome information approach would look like ideally if all of the components were in place. The service systems which have been selected to participate as exemplary sites in the System Accountability Study are quick to point out that they do not represent ideal systems. Indeed, the outcome information approaches in Texas, Pressley Ridge Schools, and Santa Cruz County, CA are being developed in the very real world of complexity and constraints which make implementing something as ambitious as outcome accountability both challenging and stressful.
Yet, these systems each demonstrate how the guiding principles for outcome accountability can be implemented in applied settings and how outcome information can be useful as a tool for improving service planning and delivery. Whether it is the Texas Children's Mental Health Plan's ability to provide regular and timely feedback about outcome information for local-level mental health authorities, Pressley-Ridge's involvement of a broad range of stakeholders in decisions about what outcomes are most important for children and families, or Santa Cruz's ability to monitor outcomes across interagency environments, the study is finding useful and generalizable strategies for developing outcome accountability.
One result of the System Accountability Study will be a monograph describing the development and implementation of these exemplary outcome information systems, the successes and challenges they have faced in this process, and an assessment of their next steps in developing and using outcome information systems in managed care environments.
In subsequent years, the System Accountability Study will provide technical assistance to service systems involved in developing approaches to outcome accountability. TA will take the form of site visits and phone consultations over a 12-month period and follow-up interviews will be conducted during the following year. Sites selected for TA will also be highlighted in the monograph resulting from this study. TA sites are currently being recruited and anyone interested in participating should contact the System Accountability Study.
Additional Outcome-Related Activities
The System Accountability Project for Children's Mental Health is a multi-faceted effort directed at increasing the use of outcomes and internal evaluation practice within child-serving agencies. Additional outcome-related efforts of the System Accountability Project for Children's Mental Health include:
- Michigan Outcome Identification Project—Twenty-three focus groups conducted with a broad range of stake-holders from inside and outside of mental health for the purpose of determining who the children's mental health system should serve, what outcomes are most important for the system to achieve, and what changes will be necessary to achieve the identified outcomes.
- Outcome Accountability Workshops—Regional workshops developed in partnership with the Judge Baker Technical Assistance Center designed to help teams from participating states organize, develop, manage, and use outcome information to improve service planning and delivery.
- Information Pathways—A special study funded by the Annie E. Casey Foundation for the purpose of identifying factors that contribute to and impede the effective use of outcome information at the local level. The product of this research will be a monograph on the use of outcome information which can guide child-serving systems in establishing strategies to better ensure the incorporation of outcome information into their decision processes.
- Journal of Mental Health Administration—A special issue focused on outcomes and system accountability in children's mental health is planned for publication in late 1997.
Mario Hernandez, Ph.D.
Sharon Hodges, M.B.A.
System Accountability Project for Children's Mental Health
Department of Child and Family Studies
Florida Mental Health Institute
University of South Florida
Tel: 813-974-4651