You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.

www.HFRP.org

The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.

Terms of Use ▼


Patricia Rogers of the Royal Melbourne Institute of Technology describes how a theory of change can provide coherence in evaluating national initiatives that are both complicated and complex.

Organizations increasingly are adopting and advocating evaluations based on a theory of change (TOC)—an explicit causal model of how program activities are understood to contribute to intended outcomes. It can be difficult, however, to use this approach when planning, managing, and evaluating programs that are both complicated and complex.

Complicated programs may have multiple parallel or complementary causal paths, that is, alternative ways of achieving outcomes; they may be multilevel programs with local, regional, national, or international layers. Complex programs have at least two main features: emergent properties, where precise objectives and implementation strategies are developed during implementation as specific opportunities develop, and disproportionate relationships, where, at critical levels, a small change can make a big difference, that is, serve as a tipping point.

Capacity-building projects are often both complicated and complex. While the project's overall goals may be clear—for example, building stronger families and communities—the specific activities and causal paths are expected to evolve during implementation; this allows programs to take advantage of emerging opportunities and learn from difficulties.

This type of project calls for a flexible theory of change evaluation. In this type of evaluation, an initial model has been developed and used to guide planning and implementation of a project; however, the theory is revised as plans change. Such flexibility, however, can make it difficult to meet the needs of a national evaluation, which requires some common framework across a set of projects that operate under common goals but are diverse in their local design and implementation. The evaluation of the Australian Government's Stronger Families and Communities Strategy provides an example of how a theory of change can adapt to meet these demands.

Australia's Stronger Families and Communities Strategy
The Stronger Families and Communities Strategy 2000–2004 (the Strategy)¹ was a national funding program that incorporated over 600 local projects across Australia. Projects were funded through seven funding initiatives, with disbursements ranging from less than $1,000 to over $1,000,000 Australian dollars. While all projects were intended to strengthen families and communities, specific project activities varied enormously.

The monitoring and evaluation of the Strategy used a TOC that addressed both the complicated and complex nature of the intervention. The theory of change (see the box) helped provide coherence across the Strategy and could be adapted for evaluating particular projects or clusters of similar projects, or for funding initiatives. This particular TOC (also called an outcomes hierarchy) delineated seven levels of Strategy outcomes, from short-term to longer term, that capacity building projects could progress through. It was developed as part of a performance indicator framework generated before projects began. Used to develop specific performance indicators, both qualitative and quantitative, for individual projects, the performance indicator framework became the basis of the evaluation framework for the overall Strategy.²

Stronger Families and Communities Strategy
2000–2004 Theory of Change


The theory of change offers seven levels of Strategy outcomes, from short-term to longer term:

Level 1: Participation, enhanced trust (via participation, networks, and new ideas) – Includes direct participation in the Strategy, including the application process, even if the application is unsuccessful

Level 2: Greater awareness, development of partnerships – Includes awareness of the Strategy, its principles, and values, as well as subject matter awareness to be developed by specific projects

Level 3: Greater choice, understanding, skills, capacity for initiative – Includes not just the particular skills, etc., that might have been the direct target of a project but also the understanding, confidence, and capacity acquired by participants in the course of planning and managing projects. Capacity would also include newly established partnerships and so forth. Greater choice could include access to a wider range of services or more appropriate services.

Level 4: Demonstration/application of greater understanding, skills, and capacity – Includes an element of sustainability beyond the completion of the funding period

Level 5: Family and community trust/resilience/adaptability – About trust that would transcend the particular project (whereas Level 1 might be about trust developed on a smaller scale through a particular Strategy project)

Level 6: Environment where communities participate in and drive their own solutions to strengthen themselves – Participation at this level transcends participation that occurs in relation to a particular project

Level 7: Stronger families and communities – About how families and communities apply the strengths from Levels 1 through 6 to improve their well-being


¹ For more information on the evaluation of the Strategy see www.facs.gov.au/internet/facsinternet.nsf/aboutfacs/programs/sfsc-evaluation.htm.
² More recently, the evaluation has developed a dynamic theory of change model that explicitly shows the feedback loops in project implementation. The model depicts how project achievements build capacity for further progress, how projects impact family and community strength in the short-term, and, through their contributions to capacity building, their impact in the long-term.

Patricia J. Rogers, Ph.D.
Associate Professor in Public Sector Evaluation
Collaborative Institute for Research, Consulting and Learning in Evaluation
Royal Melbourne Institute of Technology
Australia
Email: patricia.rogers@rmit.edu.au

 

‹ Previous Article | Table of Contents | Next Article ›

© 2016 Presidents and Fellows of Harvard College
Published by Harvard Family Research Project