The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.
Volume XIII, Number 1&2, Spring 2007
Issue Topic: Advocacy and Policy Change
End Notes
Ten Takeaways on Evaluating Advocacy and Policy Change
HFRP summarizes key observations raised in this issue of The Evaluation Exchange. Note that the focus here is on advocacy that informs public policy at the local, state, or federal levels.
1. Advocacy evaluation has become a burgeoning field. Advocacy that influences or informs policy has the potential to achieve large-scale results for individuals, families, and communities. Consequently, there is much interest in understanding how to make advocacy more effective. While advocacy evaluation was previously considered “too hard to measure,” enterprising evaluators, nonprofits, and funders are now tackling the advocacy evaluation challenge and are sharing their ideas and innovations.
2. Advocacy evaluation is particularly challenging when approached with a traditional program evaluation mindset. Evaluation approaches need to adjust to the differences between advocacy and other types of programs or services. For example, advocacy strategy typically evolves over time, and activities and their desired outcomes can shift quickly.
3. The goals of advocacy and policy change efforts—that is, whether a policy or appropriation was achieved—typically are easy to measure. The real challenge is assessing what happens along the way and what can be learned from that journey.
4. Many funders' interest in advocacy evaluation is driven by a desire to help advocates continuously improve their work, rather than to prove that advocacy is a worthy investment. At the same time, funders not currently engaging in advocacy may need examples from evaluation that convinces them of the latter.
5. Advocates must often become their own evaluators. Because of their organizational size and available resources, evaluation for many advocates requires internal monitoring and tracking of key measures rather than external evaluation.
6. External evaluators can play critical roles. In the advocacy and policy change field, external evaluators are commonly used for several purposes—helping advocates design their internal tracking systems; assessing advocates' influence on key constituencies (e.g., policymakers, media, business, voters); or assessing larger scale collaborative efforts involving multiple organizations working toward a similar policy purpose.
7. Context is important. The same result on the same measure may mean success for one advocacy effort but disappointment for another. What measures are chosen and how they are interpreted depends on the organization doing the advocacy and its experience with advocacy, the difficulty of the issue given the current policy and economic climate, and the advocacy strategy.
8. Theories of change and logic models that help drive advocacy evaluation should be grounded in theories about the policy process. This includes understanding the various leverage points and audiences that advocates may affect to move policy forward.
9. Measures must mean something. Advocates are growing sophisticated in their use of email and other social media to conduct electronic advocacy. The field needs to avoid perfunctory measures of these new techniques and make sure that measures have interpretive value.
10. Evaluation creativity is important. Assessing advocates' influence in the policy process—in particular, their influence on policymakers' (and other key constituencies') thinking and decision making—often requires methodological creativity (examples of which are contained in this issue).