The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.
Volume XIII, Number 1&2, Spring 2007
Issue Topic: Advocacy and Policy Change
Spotlight
Evaluating an Issue's Position on the Policy Agenda: The Bellwether Methodology
Policy issues need both visibility and momentum to be transformed into political action. Harvard Family Research Project's bellwether methodology helps evaluators assess if both characteristics are emerging.
Most advocacy and policy change strategies are based on the notion that getting a policy issue or proposal recognized as an “idea whose time has come” requires that it be a high priority on the policy agenda. To move a policy issue onto the “short list” of issues receiving serious attention, advocates must persuade decision makers to attend to their issue over other pressing issues vying for their attention.
Assessing advocates' success in generating the “buzz” necessary to land an issue near the top of the policy agenda is a substantial evaluation challenge. It is difficult to gauge what issues are on the agenda, much less how they got there and how they are perceived. In response to this challenge, Harvard Family Research Project (HFRP) developed the bellwether methodology.
The bellwether methodology determines where an issue is positioned in the policy agenda queue, how lawmakers and other influentials are thinking and talking about it, and how likely they are to act on it. HFRP developed the methodology as part of its evaluation of the David and Lucile Packard Foundation's grantmaking program to advance universal preschool in California. HFRP used the methodology to track whether advocacy efforts were gaining traction toward the goal of getting universal preschool onto the state-level policy agenda.
What Are Bellwethers?
By definition, bellwethers are gauges of future trends or predictors of future events. In the bellwether methodology, “bellwethers” are influential people in the public and private sectors whose positions require that they be politically informed and that they track a broad range of policy issues. Bellwethers are knowledgeable and innovative thought leaders whose opinions about policy issues carry substantial weight and predictive value.
For HFRP's evaluation, bellwethers included six groups: a) policymakers (legislative and executive), b) advocates, c) think tanks/academia, d) media, e) business representatives, and f) funders. Individual bellwethers were selected based on a range of characteristics, including their content expertise, geographic diversity, gender and ethnic diversity, partisan representation (for legislators), and cross-sector (public and private) experience.
Bellwether Interview Protocol:
|
How Does the Methodology Work?
The methodology involves structured interviews with bellwethers, but with an important twist. Instead of asking directly whether a specific issue is on the policy agenda, interview questions initially create room for a wide range of unprompted responses. Bellwethers are unaware before the interview that questions will focus on the specific policy issue of interest. They are informed about what the interview will generally cover but are not given specific details. This approach ensures that bellwethers' responses are authentic and unprompted.
For example, for HFRP's evaluation of preschool policy in California, bellwethers were aware that the interview would focus on education policy for children in California. Interviews began by asking, “Thinking about policy issues in general—not just issues related to children—what three issues or priorities do you think are at the top of the policy agenda in California right now?” Bellwethers provided a range of responses, giving a more authentic perspective on the position of universal preschool in the policy agenda. As the interview went on, questions narrowed to focus on more specific information on the policy issue of interest (see text box). HFRP also included specific questions about messaging and effective promotional activities related to universal preschool.
How Are Data Analyzed and Used?
The bellwether methodology provides both summative data about an advocacy strategy's success to date and formative data for shaping its future. Bellwether data indicate where an issue stands on the policy agenda and how effectively advocates have leveraged their access to increase an issue's visibility and sense of urgency.
Opportunities for real-time learning, including looking at competing issues and analyzing current advocacy strategy and potential areas for improvement, are also created. For example, HFRP examined what kinds of messaging and advocacy approaches made the most lasting impressions on bellwethers. HFRP also summarized bellwethers' assessments of the possibility of political movement on universal preschool in the near future and highlighted opportunities for learning by creating an opportunity for dialogue around midcourse advocacy strategy adjustments.
Elizabeth Blair
Graduate Research Assistant, HFRP
Email: blairel@gse.harvard.edu