Volume IX, Number 4, Winter 2003/2004

Issue Topic: Reflecting on the Past and Future of Evaluation

Book Review

The Success Case Method: Finding Out What Works

From time to time, we will print reviews of noteworthy new books in the evaluation field. In this, our first review, Tezeta Tulloch from Harvard Family Research Project takes a look at Robert Brinkerhoff’s The Success Case Method, a handy, accessible guide for both experienced evaluators and novices.

The Success Case Method, a new book by Robert Brinkerhoff, offers an engaging discussion of the success case method (SCM) in evaluation with step-by-step instructions on how to implement it. Brinkerhoff describes the method as a blend of storytelling and rigorous evaluation methods. SCM is, in short, a relatively quick and cost-effective way of determining which components of an initiative are working and which ones are not, and reporting results in a way that organization leaders can easily understand and believe.

The method uses the stories of individuals participating in an initiative to investigate and understand the roots of their successes and proceeds from the premise that small successes can lead to greater ones. If only 5 out of 50 participants achieve marked success, it follows that a detailed study of what these five are doing could yield instructive results for those who are struggling.

Picking out the best (and sometimes worst) accounts, verifying and then documenting them is the core of the success case method. This confirmation step is what distinguishes this approach from a simple cataloguing of positive accounts or anecdotes. Cases selected to represent an initiative’s strengths undergo rigorous scrutiny, passing muster only when respondents’ claims can be concretely confirmed. “A success story,” says Brinkerhoff, “is not considered valid and reportable until [it can] stand up in court.”

One of the SCM’s strengths is assessing what Brinkerhoff calls “soft” interventions, such as communication and other interpersonal capabilities that are generally difficult to measure. In one example, he cites an emotional intelligence training program for sales representatives obligated to cold-call customers. A success case assessment of the trainees found strong evidence that their use of trained skills decreased their fears of rejection, which led to an increase in successful calls completed, which in turn led to increased sales.

The method can also be used to estimate quickly return on investment by comparing an estimate of the dollar value of successful results with the cost of implementing the program for the participants that achieved those results. Additionally, SCM can help calculate a program’s “unrealized value”—the additional benefits a program can achieve if a greater number of people were to use the same methods employed by its more successful users. This estimate is especially useful in helping program proponents make a “business case” for improving a program that is currently underachieving.

SCM can be especially useful when integrated into a bigger effort to engineer organizational change. By quickly identifying the aspects of an initiative that are bringing about positive results, the strategy can help organizations hone in on which elements to nurture and which ones to discard before too many resources have been invested in a failing program. Alternatively, the method can be used to salvage useful parts of a program already slated for termination.

Brinkerhoff is careful to acknowledge the limits of this approach. In his discussion on conducting surveys, he notes that some (“single purpose”) surveys can take the form of a single, simple question: “Who among your staff is the most up-to-date on current office affairs?” Theoretically, anyone can handle this sort of data gathering. For those who need to elicit a wider, more diverse (“multipurpose”) range of information, Brinkerhoff recommends seeking expert advice on what can be an extremely complicated procedure.

No matter what the level of sophistication, “one major and enduring challenge for any sort of organizational analysis is trying to get key people and groups to pay attention to findings.” While there is no way to ensure a unanimously positive response, evaluators have at their disposal various reporting tactics for piquing interest. These include live presentations, video dramatizations, various report formats, and workshops for key stakeholders to discuss and apply findings. Inviting some stakeholders to participate in data collection is another way to promote investment in an evaluation’s outcomes, Brinkerhoff suggests. The book, though, offers little in the way of how those without formal experience can prepare themselves to participate actively in “data collection and analysis activities.”

Brinkerhoff’s book is bolstered by the rich selection of initiatives he draws on as examples and the charts and diagrams that illustrate essential steps—the order in which to proceed, the kinds of questions to ask, and so forth. What The Success Case Method profits from most is a lively, informal writing style that should appeal to a broad spectrum of managers, organizers, and other potential change agents, as well as evaluation beginners and experts.

Featured Book
The Success Case Method: Find Out Quickly What’s Working and What’s Not, by Robert O. Brinkerhoff. 192 pages. (2003). San Francisco: Berrett-Koehler Publishers, Inc. $24.95

Tezeta Tulloch, Publications Assistant, HFRP