The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.
|
1998 Aiming for Accountability: Lessons Learned From Eight States (digest)Diane Schilder Download a PDF of this publication (23 kb) | View all publications in this series |
Article Information
- Full Text (HTML)
- Full Text (PDF: 23 kb)
This brief draws on evidence from a study by Harvard Family Research Project (HFRP) to show that research-based accountability (RBA) can be a powerful tool to initiate and enhance systems of continuous improvement and learning. Lessons learned from the eight states in the study provide valuable insights to those responsible for RBA efforts.
About This Series
These short reports are designed to frame and contribute to the public debate on evaluation, accountability, and organizational learning.
Introduction
To reclaim citizen trust and respect and to improve decision making of public managers and policymakers, governments are increasingly turning to new forms of accountability that provide information about the results of public investments. With the implementation of the federal Government Performance and Results Act in 1993, this new form of accountability, called results-based accountability (RBA), is taking hold at the federal level. In addition, the shift to RBA at all levels of government and the increased attention to results by foundations demonstrate that RBA may be here to stay.
The expectations of RBA are that it will improve the results of government programs by:
- Facilitating collaboration among stakeholders on common goals
- Empowering frontline workers
- Reducing regulation, resulting in greater local decision making
- Engaging the public in defining solutions to society's problems
- Demonstrating the results of public investments, thereby increasing public faith in government
- Allowing policymakers to budget for results
New evidence from a recent study by Harvard Family Research Project (HFRP) shows that RBA can be a powerful tool to initiate and enhance systems of continuous improvement and learning. However, many of the expectations of RBA have yet to be met because it is still in the very early stages of implementation in most states. This research shows that government policymakers, public managers, workers, and citizens can use RBA effectively to monitor child and family services, learn what is working, change strategies accordingly, and, ultimately, improve outcomes. To be a useful tool for continuous improvement, RBA efforts must include the process of defining a vision and articulating desired results and measures, and the use of results data for program improvement and accountability.
Our research shows that if RBA efforts are to reach their potential in improving program quality and increasing accountability, they must be carefully designed and implemented. Lessons learned (see below) from the eight states in our study can provide valuable insights to those responsible for RBA efforts.
What is the current status of RBA efforts with regard to child and family services?
Most states are just beginning to design and implement RBA efforts. Even states that are touted as having the most advanced RBA efforts report that they are “about a 3 on a scale of 1 to 100.”
What factors have contributed to the current status of RBA efforts for child and family services?
Expectations are overly ambitious. People compare their progress against a set of unrealistic expectations. Unless RBA efforts are designed to promote improvement and flexibility, they will not increase collaboration, empower workers, decentralize decision making, engage the public, or provide assurances to citizens and funders that investments are producing results. Moreover, effectively linking budgets to results is a political process requiring political compromise and negotiation, which can be very difficult. Furthermore, to be implemented effectively, RBA systems require the collection and reporting of significant amounts of data which are often difficult to obtain.
RBA for child and family services is relatively new. In the areas of law enforcement, health, and education, many agencies have been collecting and reporting results data for at least a decade. By contrast, in the child and family services arena, results data have been collected and used mostly for research and demonstration projects. As such, RBA efforts for child and family services have had less time to develop compared to RBA in other areas.
Leadership and resources must be committed to the effort. Sustained commitment from bipartisan leadership is essential for the successful design and implementation of RBA efforts. Leaders must allocate funds, personnel, and time to the effort. Political and financial pressures can diminish support for the efforts over time, thus reducing the chances of successful design and implementation.
Changing thinking takes time. People who have been held accountable for following rules and regulations for their entire careers rather than producing results are not likely to change overnight. Likewise, many people have seen various reform efforts implemented unsuccessfully and are skeptical of “jumping on the bandwagon” of something that may be short-lived. As one source noted, “this is like turning a tanker in a bathtub—it takes a long time before you can see any progress.”
No single RBA model exists. While some states and localities have a longer history of RBA than others, there is no single model that can be replicated across states and localities. As such, it takes time to design and implement an RBA effort that is tailored to the unique political, demographic, and social context of a particular state or community.
What is the potential of RBA for child and family services?
While RBA for child and family services is relatively new, enough experience exists to suggest how it can best be used. In its ideal form, RBA can be a useful tool for learning and continuous improvement. To be used in this way, RBA efforts should be designed and used to:
- Engage people in defining problems and solutions to them
- Help program managers and providers plan and manage
- Demonstrate results of investments in child and family services
- Promote innovation
Challenges and solutions to the effective design and implementation of RBA efforts include the following.
Engaging People in Defining Solutions to Problems
The process of developing an RBA effort can engage policymakers, service providers, public managers, and citizens in defining problems and solutions to them. Methods to engage citizens and relevant stakeholders in this process include public polling, citizen roundtables, community forums, and meetings with key leaders and stakeholders.
Challenge: RBA efforts can increase fragmentation because different agencies, departments, and funders often have differing RBA requirements.
Solution: Those charged with designing and implementing an RBA effort should assess what other efforts currently exist and what other planning efforts are underway. Policymakers, service providers, and other stakeholders should develop a strategic plan that describes a common vision and use this to establish priorities for conflicting RBA requirements. All of those involved in RBA should adjust expectations to recognize that although complete alignment will never be possible, agreement on a general framework should be an important goal.
Challenge: Stakeholders involved in designing and implementing RBA can get “burned out” by the process since reaching consensus on goals and measures can be tedious and time-consuming.
Solution: It is important to spend time orienting stakeholders involved in RBA efforts and to set up work groups to examine technical issues. For example, Florida has developed a standard set of definitions for RBA terms, and Minnesota has developed training materials that translate jargon and explain the RBA process. Georgia and Ohio have established work groups that discuss technical details and make recommendations to those making final decisions, rather than involving all stakeholders in technical deliberations.
Helping Public Managers and Service Providers With Decision Making
The use of RBA data can help with decision making among public managers and service providers. Such data can help with monitoring progress toward achieving results. And, RBA data, coupled with program evaluation or other data, can be used to assess service strategies.
Challenge: Training, technical assistance, and data collection and reporting capacity are often inadequate.
Solution: Leveraging federal, state, and foundation resources can help ensure that training, technical assistance, and data capacity are available. The process of leveraging can be quite complicated, but some states, such as Iowa and Vermont, have been successful by using foundation dollars targeted to improve child and family outcomes to enhance RBA training and technical assistance efforts.
Demonstrating Results of Investments in Child and Family Services
To demonstrate the results of investments in child and family services, RBA data must be coupled with cost-benefit data produced through program evaluation. Nonetheless, RBA data can be used by programs and agencies to monitor progress toward achieving results.
Challenge: RBA data can be misused in order to hold public managers and service providers accountable for results over which they have no control.
Solution: To prevent misuse of RBA data, documents must indicate who is responsible for achieving results and should list factors beyond the control of public managers that could affect results. Minnesota and Oregon have included such factors in their RBA reports. In addition, training people to implement the system and use the data is essential. For example, North Carolina has developed a training process for those responsible for implementing RBA systems, and Florida has developed a training program for policymakers.
Promoting Innovation
To reach its potential as a tool for continuous improvement, RBA must be used as an incentive for innovation rather than as a punitive measure. The process of developing RBA systems can help to identify and eliminate unnecessary barriers and can promote innovation.
Challenge: Public managers and service providers may become frustrated when charged with identifying solutions to problems that they cannot implement due to constraints of existing rules and regulations.
Solution: Policymakers and public managers should create processes for systematically examining barriers to achieving results. Committees that regularly review such barriers can identify and eliminate unnecessary regulations, while ensuring the continuation of necessary regulations governing quality, access, and equity. Ohio and Georgia have both set up such committees to review and eliminate unnecessary barriers.
Challenge: Political, social, and demographic factors can impede the successful design and implementation of an RBA effort.
Solution: While no single RBA model exists, those charged with implementing RBA efforts can learn from states and communities with similar political, social, and demographic contexts. RBA efforts must be tailored to take into account state and local governance structure as well as the relative power of the governor, legislature, agencies, and county commissioners. It is important to tailor RBA efforts to the size and demographic nature of the state; what works in a small state will not necessarily work in a large state, and must be modified to ensure that key stakeholders are involved in the process.
Challenge: Adversaries can oppose an RBA effort simply for political reasons. This lack of support can jeopardize the effort.
Solution: Cultivate bipartisan champions at a variety of levels. Iowa and Vermont have both developed state committees composed of diverse stakeholders to oversee their respective RBA efforts.
Challenge: Legislative budget and appropriations committees, which are categorical in nature, may not use RBA data if they are not specifically relevant to their committee.
Solution: Target RBA reports to different audiences and ensure that those targeted toward legislators are succinct, jargon-free, and relevant to legislative committees. North Carolina has taken this into account and has tailored its RBA reports to the legislature. Make sure RBA reports are also written in a way that will be useful to constituents as they attempt to influence the legislative process. Legislators report that RBA has the greatest impact when it is referred to them by their constituents.
Challenge: Support for the RBA effort wanes after the early implementation phase.
Solution: Build relationships among key stakeholders in the state and establish trust and widespread support for the effort. The states in our study report different methods of building on these relationships, but all report that these relationships are critical to the success of the effort.
Conclusion
RBA efforts have great potential as a tool to engage policymakers and public managers in continuous improvement of child and family services. Significant challenges exist, however. To ensure that RBA efforts reach their potential, those responsible for designing and implementing such efforts must:
- Set realistic expectations
- Engage stakeholders at a variety of levels in defining problems and solutions
- Incorporate training of public managers, service providers, and citizens about the process
- Design reports to be used by a variety of audiences
- Create systems for eliminating unnecessary barriers and for facilitating innovation
- Cultivate bipartisan support and champions at a variety of levels
Free. Available online only.