Jump to:Page Content
You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.
The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.
Volume XI, Number 2, Summer 2005
Issue Topic: Evaluation Methodology
Evaluations to Watch
Patricia Rogers of the Royal Melbourne Institute of Technology describes how a theory of change can provide coherence in evaluating national initiatives that are both complicated and complex.
Organizations increasingly are adopting and advocating evaluations based on a theory of change (TOC)—an explicit causal model of how program activities are understood to contribute to intended outcomes. It can be difficult, however, to use this approach when planning, managing, and evaluating programs that are both complicated and complex.
Complicated programs may have multiple parallel or complementary causal paths, that is, alternative ways of achieving outcomes; they may be multilevel programs with local, regional, national, or international layers. Complex programs have at least two main features: emergent properties, where precise objectives and implementation strategies are developed during implementation as specific opportunities develop, and disproportionate relationships, where, at critical levels, a small change can make a big difference, that is, serve as a tipping point.
Capacity-building projects are often both complicated and complex. While the project's overall goals may be clear—for example, building stronger families and communities—the specific activities and causal paths are expected to evolve during implementation; this allows programs to take advantage of emerging opportunities and learn from difficulties.
This type of project calls for a flexible theory of change evaluation. In this type of evaluation, an initial model has been developed and used to guide planning and implementation of a project; however, the theory is revised as plans change. Such flexibility, however, can make it difficult to meet the needs of a national evaluation, which requires some common framework across a set of projects that operate under common goals but are diverse in their local design and implementation. The evaluation of the Australian Government's Stronger Families and Communities Strategy provides an example of how a theory of change can adapt to meet these demands.
Australia's Stronger Families and Communities Strategy
The Stronger Families and Communities Strategy 2000–2004 (the Strategy)¹ was a national funding program that incorporated over 600 local projects across Australia. Projects were funded through seven funding initiatives, with disbursements ranging from less than $1,000 to over $1,000,000 Australian dollars. While all projects were intended to strengthen families and communities, specific project activities varied enormously.
The monitoring and evaluation of the Strategy used a TOC that addressed both the complicated and complex nature of the intervention. The theory of change (see the box) helped provide coherence across the Strategy and could be adapted for evaluating particular projects or clusters of similar projects, or for funding initiatives. This particular TOC (also called an outcomes hierarchy) delineated seven levels of Strategy outcomes, from short-term to longer term, that capacity building projects could progress through. It was developed as part of a performance indicator framework generated before projects began. Used to develop specific performance indicators, both qualitative and quantitative, for individual projects, the performance indicator framework became the basis of the evaluation framework for the overall Strategy.²
Stronger Families and Communities Strategy
|
¹ For more information on the evaluation of the Strategy see www.facs.gov.au/internet/facsinternet.nsf/aboutfacs/programs/sfsc-evaluation.htm.
² More recently, the evaluation has developed a dynamic theory of change model that explicitly shows the feedback loops in project implementation. The model depicts how project achievements build capacity for further progress, how projects impact family and community strength in the short-term, and, through their contributions to capacity building, their impact in the long-term.
Patricia J. Rogers, Ph.D.
Associate Professor in Public Sector Evaluation
Collaborative Institute for Research, Consulting and Learning in Evaluation
Royal Melbourne Institute of Technology
Australia
Email: patricia.rogers@rmit.edu.au