The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.
Volume IX, Number 3, Fall 2003
Issue Topic: Evaluating Community-Based Initiatives
Theory & Practice
Sustainability: Does Evaluation Have a Role?
Julia Coffman and Marielle Bohan-Baker of HFRP offer ideas for the role that evaluation can play to ensure that initiative stakeholders discuss sustainability before it is too late to be useful.
Initiatives aimed at community-wide change contain a basic premise, whether articulated or not—the initiative’s work will be continued if its results warrant it Hoping for or expecting sustainability, however, does not always translate into action, and sustainability efforts are often too little too late. Typically, sustainability is considered something to be dealt with in an initiative’s later years, once some of the results are in, or when sufficient time has passed after start-up to consider questions of what should be sustained and how.
Evaluation can play a critical role in helping stakeholders start discussing sustainability early enough and maintain that discussion over time. This goes well beyond the traditional view of how evaluation can support sustainability, which is that if the results are good, data and reports can be used as marketing tools to solicit additional funding. While this function is important, evaluation can make an even more vital contribution.
Evaluation can support initiative sustainability by:
- Facilitating a focus on sustainability during strategy development
- Tracking progress and regularly feeding back information that can be used to ensure that sustainability is on course
Supporting Sustainability During Strategy Development
Sustainability should be integrated into an initiative’s strategy from the very beginning so that plans for what will happen when the funding ends are incorporated at the outset.
The strategy development process illustrated in the figure below shows evaluators and evaluation as facilitating various stages of strategy development. Evaluators, for example, can offer information in the form of a needs assessment or analysis of funding trends to support strategic analysis. They can also facilitate aspects of strategic planning, such as the development of the initiative’s theory of change or its goals and objectives. Finally, evaluators can inform strategic management by reporting back information from the evaluation itself.
This model requires evaluator involvement in the initiative from its beginning as part of the core strategy development team. It also requires that evaluators and the evaluation be flexible and be predictive of, and responsive to, the initiative and community’s needs. This approach fits well with the complex nature of most community-based initiatives, which typically evolve over time with no set script.
The model does not propose that evaluators actually make decisions about what the initiative’s strategy should be. Rather, it proposes that evaluators, who are uniquely skilled in the language and process of strategy development and often have the most comprehensive perspective on an initiative, play a supportive and advisory role in its development.
However this model also carries risks. It can compromise the evaluator’s objectivity, and is therefore not one that all evaluators subscribe to, nor one with which stakeholders may be comfortable. While too much distance from the initiative may diminish useful insight, too much involvement can cause problems. Evaluators who use this approach need to build in a set of checks and balances to help manage the risks.
Initiative stakeholders and evaluators together can support sustainability by making sure the types of questions and decisions in the table below are raised and addressed in a systematic way. For example, sustainability should be considered when determining what gets funded, how long to provide funding, which organizations are selected to participate, and what structures and supports are needed to support that sustainability.
| Strategic Analysis | Environmental Scan |
Obtain up front participants’ perspectives about their organizational and initiative-specific sustainability needs and wants. |
| Analysis of Funding Trends | Get a sense of what other funders are supporting or are willing to support and whether it matches the initiative’s focus. Identify regional or community foundations that are potential supporters. |
|
| Evidence About What Works | Include sustainability as a criterion when gathering evidence about what works. | |
| Theory of Change | Estimate how long the funder(s) need to commit to the initiative or its focus in order to make a difference. | |
| Theory of Sustainability | Determine what aspects of the initiative need to be sustained. Determine what needs to be in place to achieve sustainability. (E.g., Do long-term large grants engender dependency? Should funding be tapered?) |
|
| Strategic Planning | Goals/Objectives | Make sustainability a goal and establish objectives for getting there. |
| Initiative Participant Selection | Develop selection criteria that fit with both the theory of change and the theory of sustainability. Make expectations about the funder’s role in sustainability clear up front. Have participants include a plan for sustainability in proposals. |
|
| Initiative Structure/Tactics | Provide structure that can support sustainability (e.g., matching funds, technical assistance, funder outreach, public relations). Include grantees whose role is to provide sustainability support. Give funders a role in achieving sustainability (e.g., outreach to community foundations, institution building, spin-offs, and endowments). |
|
| Evaluation Design | Make sustainability an outcome to be tracked to feed back formative and summative information. Incorporate opportunities to report back on sustainability. Track contextual variables that will impact sustainability. |
|
| Strategic Management | Evaluation Reporting | Build in points to ask, “Does this initiative deserve to be sustained? Build in opportunities to reflect on and make midcourse changes based on what is being learned about sustainability. |
| Grantee Reporting | Develop periodic reporting mechanisms that can help grantees assess where they are in sustainability efforts. | |
Supporting Sustainability During Evaluation
Evaluation can also support sustainability by treating it as a variable to be operationalized and tracked over time, encouraging learning from an initiative’s early stages. Few evaluators take this more purposeful approach either in their evaluation design or in their reporting.
Sustainability can be thought of as a way to ensure continuation of at least four initiative aspects: (1) funding for the initiative’s organizations or projects, (2) the ideas, principles, beliefs, and values that underlie the initiative, (3) relationships that the initiative supports or encourages, and (4) the initiative’s outcomes.
Once the right focus is determined, evaluators need to operationalize sustainability so the evaluation can track it over time. Below are examples of data that evaluators might look for in the four areas. This is not a comprehensive list; the articulation of indicators to track these areas depends on the initiative being evaluated.
1. Organizations and/or Projects
- Success in obtaining additional funding
- Presence of revenue-generating strategies to support initiative-related work
- Presence of multiple funders to support initiative-related work
2. Ideas – maintaining the initiative’s core principles, values, beliefs, and commitment
- Core ideas operationalized in policies and structures
- Initiative principles applied to other projects
- Commitment to continuing work started or supported under the initiative (e.g., generation of new ideas, migration of initiative ideas and new projects)
3. Relationships
- Collaboration involving higher-order ways of working together (e.g., joint projects or products)
- Collaboration present over time (not just a one-shot effort)
- Collaboration that is not initiative-driven
4. Outcomes
- Codification of outcomes (e.g., in policy, procedures, and legislation)
- Support/demand (from public, policymakers, etc.) for outcomes
- Continued involvement/commitment of people over time
While these four areas might pertain to many initiative evaluations, they are typically not examined through the lens of sustainability. Using a sustainability lens means examining how these aspects develop over time and collecting information to determine their prospects for continuing once the initiative ends.
Evaluation’s role in supporting sustainability is ongoing, from the beginning of the initiative to the end. While these approaches will not solve the sustainability challenge, they offer ideas for where communities can find unexpected allies among evaluators.
This article is adapted from the paper, Evaluation’s Role in Supporting Initiative Sustainability.
Julia Coffman, Consultant, HFRP
Marielle Bohan-Baker, Research Associate, HFRP