The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.
Volume VIII, Number 2, Fall 2002
Issue Topic: Evaluation for Continuous Improvement
Spotlight
Moving Ahead on Results Accountability: The Human Element
Sara Watson of the Pew Charitable Trusts explains that a results accountability system must extend beyond the purely technical to also address the management of people.
When The Evaluation Exchange published its first newsletters on results accountability in the mid-1990s, federal, state, and local governments were just beginning to introduce these concepts into child and family services. Since then, researchers and practitioners in results accountability have produced a huge amount of information on how to define, measure, and collect the data necessary for accountability systems.
Hundreds of state and local government reports and websites provide extensive guidance on choosing and measuring results, including lists of results and benchmarks, language frameworks, and definitions for all of the varied categories of measures. The various taskforces that are charged with creating a results accountability system have spent years developing their own answers to the questions of what results to seek, what nomenclature to use, what numbers to collect, and how they should be collected. This level of sophistication and variety is important—without the right foundation, results accountability systems will be at the best ineffective and, at the worst, destructive.
Answering the Difficult Questions
But these are only the initial questions that must be addressed in order to use results to actually improve the well-being of children and families over time. Much harder are the questions about people, not numbers—about management, not measurement. Once the numbers are in hand, what should be done with them? Who should be accountable for what? How does one unearth the reasons behind a particular performance? What’s the best way to decide what follow-up actions will help the numbers improve? What process is both fair to all parties and effective at promoting continuous improvement? What actions will engage workers as partners and encourage them to do their best, instead of making them feel criticized and threatened?
These questions need to be asked, and answered, at the level of individual managers and workers, as well as for federal and state officials and Congressional and state legislative committees. The field needs to apply its considerable intellectual capacity to these questions if results accountability is to become embedded in the fabric of our formal and informal supports for children and families.
An Example of Addressing the Human Element
Missouri’s Caring Communities initiative is one of the earliest and best-known community-based results accountability initiatives in the country. Last year the Family and Community Trust (the organization that leads Caring Communities)¹ began a process to design a true shared accountability system, with performance targets and consequences.
One of the most important parts of this system was a formal set of responsibilities for all three of the main partners: the state agencies, the community partnerships, and the Family and Community Trust. While there was a great deal of precedence for its list of performance targets, there was much less written in the field to guide the development of shared responsibilities. One of the most important breakthroughs came when the agency staff said “we are accountable too” and backed that up with specific tasks that they would carry out to support the partnerships’ performance on results. Addressing those very human elements, at the same level or more than data concerns, was key to moving ahead with the design. It is that kind of “lesson learned” that may be helpful to other states and that needs to be explored and documented.
Assuming Accountability and Supporting Improvement
One of the biggest problems that a formal accountability system can have is a process that moves directly from collecting the numbers to follow-up actions, without sufficient intervening analysis about why a certain level of performance happened and what the next steps should be. Poor performance might be the result of:
- The wrong strategy
- The right strategy, poorly implemented or inadequately funded
- Another partner failing to fulfill its responsibility
- Some outside factor (e.g., the economy, or a population with more challenges than anticipated)
Examples of Ways to Support ImprovementPrivate reward and pressure – private conversations between those responsible for performing and those who hold them responsible Public reward and pressure – publicizing performance data and allowing public praise or inviting public inquiry Tangible rewards for success – money, equipment, etc. Increased autonomy – more freedom from certain rules or requirements (e.g., administrative paperwork) Increased assistance – voluntary or mandatory assistance following poor performance Reduced autonomy/increased oversight – decreased freedom and new or stronger oversight procedures Reduced or transferred funding – less or no financial support for poor performance (a late-stage option that must be administered in ways that protect vulnerable children and families) Changing personnel – moving people who were responsible for improvement, but not able to produce it (a last resort option and again administered with care) Source: Watson, S. (2000). Informed consent: Advice for state and local leaders on implementing results-based decisionmaking. Washington, DC: The Finance Project. Available at www.financeproject.org/ |
Knowing which reason is the correct one is essential to deciding on next steps. The box on the right lists possible follow-up actions—both “carrots” and “sticks”—for different levels of performance. They are based on choices states and localities are currently using, and can be applied at the individual and organizational levels.
It is immensely challenging to decide on and implement follow-up actions that address the correct problem, hold people responsible for the appropriate level of performance (e.g., don’t penalize individuals for performance on results that are much broader than they can hope to affect and vice-versa), increase trust, improve performance, and satisfy the public and the media that progress is being made. Making the right decisions about who should be accountable to whom for what is crucial. Mistakes here can doom accountability systems by raising fears and resistance among those who are responsible for improvements.
Most accountability systems are just beginning to complete the cycle of collecting data, analyzing the reasons behind performance, and taking action to improve. Just as experts and practitioners have produced hundreds of reports and guidebooks analyzing different definitions of data variables, we need commensurate attention to who should do what based on the data. Only then will results accountability systems fulfill their potential of improving outcomes for children and families.
Sara Watson
Program Officer, Education
The Pew Charitable Trusts
2005 Market Street, Suite 1700
Philadelphia, PA 19103
Tel: 215-575-9050
Email: swatson@pewtrusts.com
The opinions expressed in this article are those of the author and do not necessarily reflect the views of the Pew Charitable Trusts. Past articles on results accountability can be found in The Evaluation Exchange Vol. IV, No. 1, 1998, Vol. II, No. 3, 1996, and Vol. II, No. 1, 1996. Click here to view archived issues.
¹ As this issue was going to press, the state was disbanding FACT. However, the experience was still important in developing an accountability design accepted by all the partners.