Volume XIII, Number 1&2, Spring 2007

Issue Topic: Advocacy and Policy Change

Spotlight

Evaluating Advocates' Spheres of Influence With Domain Leaders

The evaluation of the Center for Tobacco-Free Kids gathered data from a wide range of audiences that the advocacy organization targets in order to influence public policy.

Evaluators face a difficult challenge in assessing the effectiveness of advocacy campaigns designed to influence policy. Methods must extend beyond traditional opinion polling and quantitative measurement to capture the qualitative influence that advocacy activities have on policy leaders' thinking and positions. Evaluators also must consider the broad range of audiences and domains that advocates target to inform policy. Advocacy typically involves much more than attempts to reach elected officials directly; it also involves working strategically with the many groups and individuals who play roles in and influence the policy process.

Several years ago, the Robert Wood Johnson Foundation (RWJF) commissioned The Lewin Group to evaluate a key RWJF grantee, the Center for Tobacco-Free Kids1 (the Center), a national advocacy leader and resource on tobacco control. Begun in 1995, the Center attempts to influence public attitudes and policies on tobacco for the purpose of preventing children from smoking, helping smokers to quit, and protecting everyone from secondhand smoke. The Center promotes the visibility of tobacco issues in the media, educates policymakers, and partners with state and grassroots advocacy groups to inform policy at the state and federal levels.

The evaluation's goal was to investigate the Center's role in tobacco control advocacy, its strengths and weaknesses, and opportunities for future growth. In response to the challenges posed by evaluating advocacy efforts, The Lewin Group and RWJF developed an evaluation approach that examined questions about the Center with domain leaders connected to tobacco control issues.

Who Are Domain Leaders?
Like most advocacy organizations, the Center works with leaders in a range of ways and across a variety of domains—including government, media, research, and state and national advocacy. Individuals and organizations working in these domains are key players in the policy process, influencing the policy agenda and each other in interconnected ways. From these domain leaders, The Lewin Group sought to capture perspectives on the Center's effectiveness.

The evaluation targeted domain leaders who were connected to tobacco control efforts. The Lewin Group selected these leaders—who were often aware of the Center's work and who included Center critics—based on expert key informant suggestions. Domain leaders in this evaluation included:

• Senior federal officials working on tobacco issues
• Journalists covering tobacco issues
• National tobacco control advocates
• State tobacco control advocates
• Researchers focused on tobacco issues
• RWJF staff focused on tobacco issues
• Other Center funders
• Center board members and staff

Methodology
The evaluation approach was developed with guidance from RWJF staff and an expert panel of advisors that included three substantive domain experts who worked closely with The Lewin Group on the evaluation. Almost 70 semistructured telephone interviews were conducted with domain leaders. The three substantive domain experts conducted interviews with federal policymakers and state advocacy groups; senior Lewin Group staff conducted all others. Interview participants knew that the interview's purpose was to talk about the Center's work. Questions covered:

• Familiarity with and nature of interactions with the Center
• Sources for accessing tobacco information
• Utility of the Center's services and information resources
• The Center's perceived contributions and accomplishments
• Recommendations for Center improvements

All leaders were asked to describe their interaction with the Center and to provide feedback on ways the Center might improve both its relationship with the domain leaders and the Center's influence in the broader policy arena. Protocols for each group differed but featured some common questions so that responses could be compared across domains. These extensive interview protocols, which included specific questions about the Center's activities, provided rich data about the Center's positioning in the tobacco control field.

Data were analyzed thematically both within and across domains. Findings were reported separately for government officials, media, state advocates, national advocates, researchers, and RWJF staff.

Advocates typically target multiple leverage points in the policy process. Their effectiveness with audiences at each point can make a substantial difference in their ultimate effects on policy. By seeking the full array of domain leader perspectives in the Center for Tobacco-Free Kids' intended sphere of influence, the evaluation was able to contribute data on multiple facets of the Center's advocacy strategy, including media outreach, research, and coalition and constituency building.

1 The Center also is known as the Campaign for Tobacco-Free Kids.

Nancy Fishman
Senior Program Officer
Robert Wood Johnson Foundation
P.O. Box 2316
Princeton, NJ 08543
Email: nfishma@rwjf.org

Rick Harwood
Vice President
The Lewin Group
3130 Fairview Park Drive, Suite 800
Falls Church, VA 22042.
Tel: 703-269-5500.
Email: rick.harwood@lewin.com