Volume XIII, Number 1&2, Spring 2007

Issue Topic: Advocacy and Policy Change

Ask the Expert

What does monitoring and evaluation look like for real-life advocates?

Stephanie Schaefer, codirector of research at Fight Crime: Invest in Kids—a national nonprofit, bipartisan organization of law enforcement leaders and violence survivors—describes how they use evaluation to inform their advocacy and demonstrate their impact.

Can you describe your advocacy work?

We promote investments in children's programs and policies (e.g., quality early childhood education, child abuse and neglect prevention, after school, and interventions for troubled kids) as a way to prevent crime and violence. Our members include more than 3,000 police chiefs, sheriffs, prosecutors, police leaders, and violence survivors. Supported by our Washington, DC-based office and 10 state offices around the country, these members are our primary spokespersons, and they advocate at the state and federal levels.

Our national and state offices work in four areas—strategic membership recruitment and education, research and policy analysis, public education earned media campaigns, and the education of policymakers. These activities support and inform our advocacy.

How does Fight Crime: Invest in Kids evaluate its advocacy efforts?

Our funders are interested in accountability, as are we. Funders want to know how their support is making an impact. That can be a challenge with advocacy, as we are one influence among many in the policy process. But we appreciate funders' interest in accountability because the evaluation of our efforts ultimately increases their awareness of how advocacy can support social change goals.

We think about our evaluation in two ways—occasional point-in-time evaluations conducted by external evaluators and ongoing tracking that we do internally. Early on (Fight Crime: Invest in Kids started in 1996), we had an extensive external evaluation, supported by the William T. Grant Foundation, which examined our work in the broader policy environment. For example, evaluators talked to policymakers about their awareness and use of our crime-prevention research and message. We found that feedback valuable and used the process to inform development of the measures that we now track internally.

Our quantitative measures are linked to different aspects of our advocacy. For example, with regard to our member education and training, we track the number of contacts our law enforcement members make with their members of Congress. For our media outreach, we track newspaper, radio, and television coverage generated by press conferences or report releases. We also track growth in our membership, presentations, endorsements of our policy positions by law enforcement associations, and website statistics that give us a sense of our electronic visibility and exposure.

We consider our measures individually and as a whole to determine what they say about our overall progress toward our mission of giving kids the right start in life. Each year, we track a set of key indicators linked to our organizational capacity and impact, including increases in government investments in kids that reduce crime. We've tracked these indicators since our organization started and use them to assess our growth.

In addition to quantitative measures, we document our impact qualitatively through narrative stories. We recognize that these stories are anecdotal but that, combined with our quantitative measures, they provide context and convey more vividly how we and other advocates make a difference. For example, we document when policymakers quote our language or use our messages in public debates on the Senate or House floor. Most people understand that policy change doesn't occur in a vacuum, and this can be one way to make advocacy more concrete.

How do you use the data you collect?

First, we use the data to inform our strategy and determine which areas to emphasize more. For example, we use member data to identify when and where to increase our member recruitment and mobilization capacity. Or we use data on our print and television coverage to tell us whether our media strategies in targeted media markets are working. We also use data on our members' meetings with key policymakers to assess whether our members are getting the policy message to them.

Second, we use the data to demonstrate our progress and impact. For instance, we use data to make a case about the impact that advocacy has on federal investments and policy and about the specific value we contribute. By doing this, we can show foundations that supporting advocacy and policy work can impact the lives of far more children than funding direct services alone.

In the past, we've tracked foundation investments in federal after school advocacy by gathering data from foundations investing in this issue and the key advocacy organizations they fund. We then add data on federal after school funding for the same timeframe to see how well the tra-jectories map. Finally, we add data about our own after school work and again examine the patterns. Of course, we recognize the interpretive limits of these comparisons and that we are one of many influences that may be affecting investment levels, but such an analysis makes a solid case about the importance of nonprofit advocacy's contribution to policy outcomes.

HFRP Staff
Email: hfrp_pubs@gse.harvard.edu