The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.
Volume XIII, Number 1, Spring 2007
Issue Topic: Advocacy and Policy Change
Promising Practices
The eNonprofit Benchmarks Study: Diving Into Email Metrics
Karen Matheson from M+R Strategic Services describes a recent study that helps nonprofits measure and interpret their online advocacy and fundraising success.
In the for-profit dot-com world, the bottom line is easy to measure—it all comes down to dollars and cents. For nonprofits, success is more difficult to define. How many people were educated? Informed? Served? Engaged? Activated? How much money was raised? Did legislative policy change? Did corporate policy shift? Was public opinion swayed?
The eNonprofit Benchmarks Study is the first of its kind to look at the effectiveness of major American nonprofits using the Internet to raise money and influence public policy. Nonprofits can use this study to measure and compare their online performance to other organizations.
Methodology
The study provides a snapshot of key metrics and benchmarks for nonprofit online communications, including email fundraising and advocacy. To develop these metrics, M+R Strategic Services analyzed data from three sources: (a) nonprofit study partners, 15 key national nonprofits in the environmental, civil/legal rights-based, and international aid sectors, which had substantial online communications and marketing programs; (b) aggregate data from Convio, GetActive Software, and Kintera, major providers of online communications tools for nonprofits; and (c) an online survey of the broader nonprofit community with 85 respondents.
Most data came from drilling down into hundreds of email messages sent by the 15 study partners to their email list members over 2 years—from September 2003 to September 2005. We coded statistics for these messages by nonprofit type (environmental, rights-based, or international aid) and then sorted them into message-type categories (including advocacy, fundraising, e-news, and other).
The study has chapters on return on investment, email messaging, email list growth, email list composition, online advocacy, and online fund-raising. One of the study's most revealing chapters—with regard to measuring the quality and effectiveness of announcements to nonprofit organizations' email subscriber lists—is on email messaging metrics.
Email Messaging Metrics
The table titled “Nonprofit Advocacy and Fundraising Email Metrics” shows the key email metrics identified for advocacy and fundraising based on the 15 nonprofit partners in the study. Metrics are listed first for all partners, and then for the nonprofits in each sector examined. The email metrics and how they were calculated are described below.
Open rates generally are an indicator of three factors: (a) how engaging an email's subject line is, (b) the email list's quality, and (c) the strength of the relationship between the organization and its subscribers. The eNonprofit Benchmarks study calculated open rates by dividing the number of people who opened an email message by the total number of recipients. The study found that open rates for advocacy emails dropped significantly across the board in the 2 years covered by the study. However, there were no corresponding drops in page completion or response rates.1
Page-completion rates assess the quality of an advocacy or fundraising online form. This metric was calculated by dividing the number of people who clicked on the link that sent subscribers to a form by the number of people who actually completed it.
Click-through rates measure an email's persuasiveness and salience. Although there are several ways to calculate click-through rates, the study calculated this metric as the number of people who clicked at least one link in the email divided by the total number of message recipients. This calculation is most useful for messages such as online newsletters where the goal is motivating subscribers to click on one or more article links.
Response rates gauge an email message's overall success. The study calculated this by dividing the number of people who completed on online form by the total number of email recipients.
In the email messaging metrics chapter, as in its other revealing chapters, the eNonprofit Benchmarks Study provides nonprofit organizations with new and practical ways to define success. To learn more about email messaging metrics or to read the full report, visit www.e-benchmarksstudy.com.
|
All Partners
|
Environmental
|
Rights
|
International
Aid |
||
| Open Rate | Advocacy | 26% | 26% | 25% | 26% |
| Fundraising | 23% | 22% | 23% | 23% | |
| Page-Completion Rate | Advocacy | 84% | 91% | 81% | 79% |
| Fundraising | 22% | 32% | 7% | 33% | |
| Click-Through Rate | Advocacy | 9% | 13% | 7% | 8% |
| Fundraising | 1.5% | 0.8% | 2.1% | 1.7% | |
| Response Rate | Advocacy | 10% | 14% | 7% | 7% |
| Fundraising | 0.3% | 0.2% | 0.2% | 0.6% | |
| 1 Some click-through rates are higher than response rates. This is because messages in the click-through pool included all the links (not just those that lead to an advocacy action or donation form), while the messages in the response pool included only messages in which the link led to a fill-out form. | |||||
1 In the fall of 2006, M+R Strategic Services did further research on the decline in open rates. Research is available at www.mrss.com/news/Why_Open_Rates_Are_Dropping_M-R_Strategic_Services.pdf .
Karen Matheson
Manager
Quantitative Research and Analysis
M+R Strategic Services
615 Second Avenue, Suite 550
Seattle, WA 98104
Tel: 206-447-9089
Email: kmatheson@mrss.com