The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.
Volume VIII, Number 3, Winter 2002
Issue Topic: Public Communications Campaigns and Evaluation
Spotlight
Beyond the Usual Suspects
Julia Coffman, from Harvard Family Research Project, describes methods for campaign evaluation that are unique to the communications arena.
When thinking about methods to assess communication campaigns, the usual suspects come to mind—focus groups, interviews, surveys, and polling. Indeed, these methods are critical, and most campaign evaluations use them. Yet as campaigns become increasingly sophisticated both technologically and strategically, evaluators may want to explore methods of tracking and assessment used by media and communication researchers that are lesser known among program evaluators.
Media Tracking
Many services are available to offer reports of a campaign’s coverage or an ad’s placement in print and on television, radio, and the Internet. They can also return data on a campaign’s exposure, using commercial advertising measures, such as the number of media impressions1 or gross rating points.2
Print media tracking may offer data on the volume of coverage generated (e.g., column inches), messages sent versus messages placed, or content analysis of how often coverage reflects the campaign’s messages.
Broadcast tracking can follow special encoding on a public service announcement (PSA) or video news release sent out through satellite uplink and return information about its downloading and airing. Services like Nielsen’s Sigma Service track what stations air the ad, the airdate and time, the broadcast market, rank compared to other downloads in that market, and the estimated audience size.
Web or Internet tracking services can provide data hits and navigation patterns for an entire website or individual pages, and can offer reports on where banner ads run.
Ad Assessments
Ad assessments can provide measures of ad recognition (aided) and recall (unaided). A Starch Readership Study conducted by the company Roper ASW, for example, provides measures based on what ads readers of a specific publication have seen and read. The study is based on a minimum of 100 interviews conducted one to three weeks after a publication is released and at locations throughout the publication’s distribution area. It delivers three scores: percentage of readers who remember seeing the ad, percentage who recalled the name of the advertiser or campaign, and percentage who read half or more of the ad. It also provides benchmarks of the ad’s scores by comparing it to all other ads in the same publication, and to other ads in its same category.
Related Resources Media Tracking Ad Assessments Framing Analysis Social Network Analysis |
Framing Analysis
Framing analysis examines how issues are presented or discussed (framed) in the media. It looks for key themes, expressed as arguments, metaphors, and descriptions to reveal which parts of the issue are emphasized and which are missing. Campaigns that attempt to influence an issue’s media coverage may conduct pre and post framing analyses (after a sufficient amount of time has passed).
Social Network Analysis
Network analysis can be used to examine patterns of interpersonal communication as a campaign outcome3 and as an independent variable influencing behavior.4 For example, a campaign might attempt to affect who individuals turn to in order to get advice on a topic such as contraception or nutrition—aiming to decrease reliance on one’s friends and increase reliance on people who are more knowledgeable about the topic. Network analysis can determine whether the campaign affected patterns of who people turn to for advice. While its value as an evaluation tool can be great, network analysis has been used infrequently in practice because the techniques are not well understood.5
Julia Coffman, Consultant, HFRP
1 The number of media impressions represents everyone who might have seen an ad or campaign. It often runs into the hundreds of thousands or millions.
2 A campaign that buys media time can purchase a certain number of Gross Rating Points (GRPs). A campaign that attempts to earn media time can calculate how many GRPs were donated. If 1% of the target audience sees an ad once, the ad earns 1 GRP. A weekly score of 100 GRPs means that an average person in the target audience saw the ad one time that week. A weekly score of 500 GRPs means that an average person in the target audience saw the ad five times that week. GRPs are estimated based on the projected audience for a particular medium. These projections are based, for example in television, on Nielsen ratings.
3 Valente, T. W., & Saba, W. (2001). Campaign recognition and interpersonal communication as factors in contraceptive use in Bolivia. Journal of Health Communication. 6(4), 1-20.
4 Valente, T. W., & Saba, W. (1998). Mass media and interpersonal influence in a reproductive health communication campaign in Bolivia. Communication Research, 25, 96-124.
5 For an introduction to the methodology, see Dr. Thomas Valente’s 1999 publication, Network Analysis in Public Health. Email the author for a copy at tvalente@usc.edu.