Volume VII, Number 1, Winter 2001

Issue Topic: Strategic Communications

Theory & Practice

Pitching Policy Change

In previous issues of The Evaluation Exchange, we have written about the development and evaluation of such complex projects as comprehensive community initiatives.1 Many of these endeavors are developed with broad-based change in mind and may be both multi-site and multi-level. In this article, we present some of the challenges voiced by communications experts in interviews about the use and evaluation of mass media initiatives.

With growing demand for accountability and fierce competition for funding, nonprofit organizations and foundations are increasingly looking for ways to use information to their advantage. To a great extent, the capacity of social sector organizations to achieve their visions and goals lies in how effectively they communicate. Whether proposing a plan to a board of directors or initiating a large-scale media campaign, how programs and initiatives develop, package and deliver information impacts their chances of success. Hence, strategic communications has become an organizational function that is difficult to live without. Knowing that the media's influence is not likely to wane in the near future, we believe more work is needed to guide the development and evaluation of initiatives. Based on investigations into the literature and interviews with communications experts, this article presents our understanding of the role of strategic communications in nonprofits and the challenges faced by those charged with evaluating campaigns.

Frank Karel, vice president for communications at the Robert Wood Johnson Foundation, defines strategic communications as “the managed process by which information is produced and conveyed to achieve specific objectives vital to an organization's mission.”2 While this definition sounds simple, it encompasses a broad range of activities—from grassroots organizing and coalition-building to local and national campaigns using mass media to convey messages to the public. For the purposes of this article, we focus on the use of mass media initiatives to deliver public will-building messages and the importance of incorporating evaluation into the process of planning and implementing all aspects of a strategic communications campaign.

Most social sector organizations deliver more than just a service; they strive to improve people's lives. A social change mission is also at stake, requiring the use of communications strategies to generate public demand and increased investment in initiatives and programs. While it is indeed a challenge to produce and convey information that increases the likelihood that an organization's social change mission will be accomplished, even more complex is how that process and its intended effects in the political realm are evaluated.

Large scale strategic communications initiatives with a mass media component also reflect complexity in at least three ways. First, they often connect and integrate different types of activities that are required for broad-based change. Second, they operate within a context in which it is impossible to clearly link cause and effect. Last, and perhaps most importantly, strategic communications cannot be divorced from an organizations' overall goals and objectives.

An example of this kind of complexity is the California Children and Families Commission (CCFC), which is responsible for the implementation of the California Children and Families First Act (Proposition 10) and is charged with implementing a systems-change strategy to ensure that children ages birth to five receive the services they need. In keeping with their mission, they have identified five sequential building blocks necessary to achieve their mission and goals.3 Two of these building blocks “Changing Public Awareness: Public Education and Outreach” and “Improving Public Policy: Policy Development and Advocacy,” each have their own strategy but are not divorced from overall CCFC mission and goals.

Who Sets the Public Agenda?

During the early to mid-20th century, scholars such as Walter Lippman and Bernard Cohen pioneered the early thinking on what we now know as “agenda setting.” Lippman observed that the media give priority to certain issues over others, essentially acting as a gatekeeper between the world's events and what the public actually sees. Furthermore, Cohen observed that the press was not necessarily adept at telling people what to think, yet was successful in telling people what they should think about, ultimately setting the public agenda.4 Paul Lazersfeld demonstrated the impact of the media from another perspective, suggesting that the media often influences only a few opinion leaders or “elites” who, in turn, impart their knowledge to the public.5

Building on this base, a number of complex time series analyses have been conducted more recently that show that public opinion seems to follow, not lead, the agenda set by the press.6 Moreover, experiments by Shanto Iyengar and David Kinder show that the public's concern about issues and the importance they assign to those issues follows quickly on the tail of the attention paid to those issues by the national media.7

But how often the media, and television news in particular, presents an issue can and does have an impact on the attitudes and responses that are elicited from the public.8 The more attention that the media pays to an issue or candidate, the more that issue is primed. How an issue is framed—the subtle differences in the presentation of a question, statement or problem that will elicit different responses and attitudes—also affects how people come to their opinions. The importance people attribute to social issues depends on a complex and subtle combination of elements, including problem and solution presentation and the identity of the message's deliverer.

While working to influence what lands on the public agenda requires a well-articulated and implemented strategy, those using the mass media to accomplish policy goals need to understand the factors that impact what lands on the policy agenda. The agenda-setting process is now thought to encompass media, public, and policy agenda-setting.9

Although factors such as interest groups impact what ultimately appears on the policy agenda, evidence has shown that “public opinion is often a proximate cause of policy, affecting policy more than policy influences opinion.”10 Given that public opinion is such a strong predictor of policy makers' decision making, it is not surprising that mass media initiatives are used by organizations as a method to achieve their larger organizational goals. The powerful influence of the news media, particularly television news, provides the incentive to develop activities that will influence their coverage of issues. Yet the convergence of multiple factors that impact the ability to achieve desired outcomes can create complexity and confusion for those seeking to plan, implement, and evaluate such an initiative.

Influencing the Public Agenda

Designing and Evaluating Mass Media Initiatives
Some media campaigns aimed at changing certain attitudes and behaviors have shown some results; however, changes in public opinion and policy are a challenge to measure.11 However difficult it is to link cause and effect in the evaluation of mass media initiatives, it is important to invest in understanding the role that the initiatives play in shaping public will and policy.

Evaluators of mass media initiatives face significant challenges. Those engaging in projects aimed at influencing public will need to show that the investments made in strategic communications have been well spent while funders and their boards want to know how media efforts contributed to larger institutional and programmatic goals. How can these initiatives be approached in a way that takes into account the many activities required, the multiple media channels used (news, advertising, entertainment) and the organizational and political contexts within which the initiative operates?

Articulating a Complex Strategy and its Outcomes
At a minimum, when considering a media initiative, it is critical to articulate why it is needed and what purpose it serves. Time should be taken at the outset to articulate what the strategy will be—desired outcomes, how these outcomes and the initiative reflect and promote the mission of the organization, how key outcomes will be achieved and what contextual factors may impact that process.

Reflecting on our discussion of agenda-setting, a long-term goal, such as policy change or an increase in public investment, may take years to accomplish. In this era of accountability, there will undoubtedly be pressure to show results much earlier. Building a case along the way—with demonstrable and measurable intermediate outcomes—that strategic media interventions targeted at public opinion contributed to a change in public will and policy is key to demonstrating the value of mass media initiatives and strategic communications campaigns.

Therefore, knowing what outcomes are needed one year from now—intermediate outcomes—to make other outcomes possible two to five years out is an important aspect of developing a logical and coherent strategy. If the purpose of the mass media initiative is to ultimately increase public funding for a particular cause (long-term outcome), what must be accomplished in the interim to make that possible? Whose awareness needs to be raised? Which opinion leaders must be convinced of the validity of the cause? And how will these challenges affect an organization that is trying to accomplish these objectives?

Matching the Outcomes to the Mass Media Initiative Strategy
In evaluating complex mass media initiatives with multiple variables, there may be a tendency to go with the “usual suspects” in terms of outcomes, as suggested by communications expert Susan Bales, who raises the point that evaluators often tend to measure the “measurable.” Counting messages delivered without clearly thinking through what the right outcomes and appropriate approaches are for measuring desired outcomes substitutes outputs for outcomes. These include outputs such as the number of op-eds placed in the paper, or the number of pamphlets distributed. Although outputs may be important, they are not replacements for outcomes—quantity does not necessarily mean quality. Measuring process is important too; one useful approach may be to look at whether op-eds supported a reframing of the issue consistent with the initiative's mission.

Public opinion is a common outcome measured by evaluators of mass media initiatives. Before embarking on an evaluation procedure to measure public opinion, a critical question to ask is whether the mass media initiative is designed to make great shifts in public opinion on an issue or merely get the issue on the public agenda, providing information to encourage new ways to think about the issue. Which path the initiative chooses—moving public opinion or informing an audience—has significant implications for the specific goals, objectives and outcomes that are articulated early on, and therefore, what outcomes are ultimately measured.

The issue of early care and education illustrates an approach that highlights how encouraging new ways of thinking about an issue can be used as a strategy to spur future investment. As seen in recent polls and as a result of being exposed to important findings from recent brain research, the public acknowledges that learning begins at birth; the public does not, however, always connect these developmental findings with the need for quality early care and education. Furthermore, the public does not always support increases in early care and education funding if they perceive that funding will be shifted from primary school education.12 Thus, strategists are now considering how to reframe the concept of early care and education so it is viewed as part of a continuum of learning that starts at birth and proceeds through elementary school. Strategists hope that, by connecting early care and education more closely with the concept of primary education, public support of increased investment for education during the birth to five age span will grow.

Applying Information During Planning and Implementation
With the strategy and outcomes articulated, the process of planning for and implementing a mass media initiative should acknowledge the value of applying research-generated information at the start of planning and during implementation with the help of the formative evaluation process. Overall, evaluating complex mass media initiatives and strategic communications campaigns is not very different from evaluating large, complex program and policy initiatives. As in any complex evaluation, it is important to be aware of the cycle of evaluation in which formative research—such as needs assessments or market research—leads to intervention design, which is accompanied and followed by evaluation and course correction and re-design for ongoing continuous improvement.

There is much to be learned from those in the field who are implementing communications initiatives as well as from the different approaches to evaluation that have been applied to other complex initiatives. With a call for evaluation designs that foster learning among stakeholders and with the ever-present accountability demands that programs, evaluators and foundations face, the need for clear thinking on this topic has never been greater.

1 See Coffman, J. (2000). Simplifying complex initiative evaluation. The Evaluation Exchange, 5(2/3), 2–3, 15. Cambridge, MA: Harvard Family Research Project.
2 Karel, F. (2000). Getting the word out: A foundation memoir and personal journey. In S. L. Isaccs & J. R. Knickman (Eds.), To improve health and health care 2001: The Robert Wood Johnson Foundation anthology. Princeton, NJ: The Robert Wood Johnson Foundation.
3 See the California Children and Families Commission's 2000–2001 Fiscal Year Objectives and Priorities document at http://www.ccfc.ca.gov/PDF/ObjPriorities7-20-00.pdf for more information. Proposition 10 is funded by a tobacco tax that generates approximately $700 million annually providing major noncategorical resources to help California's young children and their families.
4 Rogers, E. M. (1994). A history of communication study: A biographical approach. New York: The Free Press.
5 Ibid.
6 Iyengar, S., & Kinder, D. (1987). News that matters: Television and American opinion. Chicago: University of Chicago Press.
7 Kinder, D. R. (1998). Communication and opinion. Annual Review of Political Science, 1, 167–197.
8 Gilliam, F. D. (1999). The influence of local television news frames on attitudes about children. In Effective language for communicating children's issues. Washington, DC: Coalition for America's Children with the Benton Foundation.
9 Rogers, E. M. (1994). A history of communication study: A biographical approach. New York: The Free Press.
10 Page, B. I., & Shapiro, R. Y. (1983). Effects of public opinion on policy. The American Political Science Review, 77, 175–190. Also see Weiss, J. A., & Tschirhart, M. (1994). Public information campaigns as policy instruments. Journal of Policy Analysis and Management, 13, 1, 82, 119. Association for Public Policy Analysis and Management.
11 Dungan-Seaver, D. (1999). Mass media initiatives and children's issues: An analysis of research and recent experience relevant to potential philanthropic funders. Prepared for the McKnight Foundation. Also see Andreasen, A. R. (1995). Marketing social change: Changing behavior to promote health, social development, and the environment. San Francisco: Jossey-Bass Publishers.
12 Voices for Illinois Children commissioned Market Strategies, Inc. to do a poll of 600 registered voters in Illinois between February 15 and 19 of 2000. The margin of error on the poll is +/- 4.0%. Source: www.earlycare.org. See also Bostrom, M. (1999). Children as a political issue: A review of public opinion. In Effective language for communicating children's issues. Washington, DC: Coalition for America's Children with the Benton Foundation.

Marielle Bohan-Baker, Research Associate, HFRP