Search Publications |  The Evaluation Exchange |  FINE Network |  OST Database

HFRP > Evaluation > The Evaluation Exchange > Issue Archive > Children and Youth > Building Local Capacity for Evaluation: The Michigan Abstinence Partnership Evaluation

Volume V, Number 1, 1999

Issue Topic: Children and Youth

Evaluations to Watch

Building Local Capacity for Evaluation: The Michigan Abstinence Partnership Evaluation

Stacy Constantineau Meade of Michigan Public Health Institute writes about the increasingly important role of evaluation in enabling communities to promote and sustain change.

In recent years, evaluation has come to play an increasingly important role in enabling communities to promote and sustain change. Various methods have been used to assist communities in this endeavor, most notably, the empowerment model of evaluation. Ultimately, this model assists communities in evaluating themselves in a constant interchange between program planning and evaluation. For the last five years, the Michigan Abstinence Partnership (MAP) Evaluation Project has utilized an empowerment evaluation approach to increase capacity for evaluation for community coalitions throughout Michigan. This article discusses the evaluation design and provides a glimpse at some of the lessons learned from implementing this model.

Background

MAP is a statewide, public health initiative based on the belief that teaching abstinence to children aged nine to fourteen can help to promote positive health behaviors and reduce risk-taking behaviors later in life. The impetus for the development of MAP came from the Michigan Department of Community Health (MDCH) in 1993.

Formation of MAP stressed recruitment of a broad base of representatives in the community. Members of MAP include public health officials, educators, medical and health care providers, the religious community, parents, community leaders, child advocates ranging from Planned Parenthood to Michigan Right to Life, and businesses throughout the state.

Community coalitions direct and facilitate MAP programming at the local level. They are located in both rural and urban settings. Each community is encouraged to tailor programming to meet the individual needs of its population.

Evaluation Design

MAP stresses both statewide and local community-level evaluations. At the local level, community coalitions develop Community Action Plans that incorporate the evaluation of activities on three levels.

• First, process evaluation is used to examine how program activities were implemented and how they can be improved in the future.
• Second, the short-term impact on participants’ knowledge, skills, attitudes, and behavior is measured.
• Third, the long-term outcomes of program activities will be determined through a time-series analysis of community indicators.

The MDCH and the Michigan Public Health Institute, a nonprofit research organization, joined efforts to devise a plan to provide evaluation support to local community coalitions. The primary goal of this project is to increase each community’s capacity for evaluation while producing measurable outcomes for program activities. To accomplish this goal, an empowerment evaluation model was selected with a view toward coalition self-sufficiency. The project provides technical assistance in evaluation to community coalitions through workshops, individual site visits, regional technical assistance meetings, evaluation resource materials, and regular contact via fax and phone. Technical assistance to community coalition staff is provided in the following areas:

• Evaluation planning, including the development of evaluation designs
• Developing goals and objectives
• Determining outcomes and outcome measures
• Developing quantitative and qualitative evaluation instrumentation
• Evaluation implementation, including data collection and analysis
• Reporting and utilizing evaluation results.

Lessons Learned to Increase Community Capacity for Evaluation

Throughout the five-year Evaluation Project, several lessons learned have emerged from the activities of the project:

• Evaluation technical assistance must be tailored to the distinct cultural and political contexts of each community coalition. Each community exists in unique cultural surroundings. The technical assistance and evaluation needs of a Native American tribal school, for example, are very different from those of an urban teen center. The project staff must also be sensitive to politics that exist within each community. Sensitivity to these factors contributes to the building and maintenance of trusting working relationships.

• Communities must be met where they are. This is true both figuratively and geographically. Communities exist across a continuum of capacities and resources. Technical assistance must be tailored to meet individual needs. The empowerment evaluation model allows the evaluation to adapt on an individual level to the needs and expectations of the community. It is also important to consider where communities are geographically. Travel to each coalition site is invaluable because site visits allow the evaluation staff to witness firsthand the work surroundings, staff dynamics, and program implementation.

• Networking with other community coalitions is important. As the Partnership has evolved, the communities have gained new insights and ideas from their colleagues around the state. Networking has emerged as an important theme in the evaluation. At trainings and workshops, adequate time is scheduled into the day to insure an opportunity for discussion and sharing.

• Evaluation must be incorporated into the Community Action Plan. Community coalition programming and evaluation are more successful when planned early. The evaluation staff support coalitions with the development of program goals, objectives, and instruments through technical assistance before programs are implemented.

• Coalitions should be encouraged to take more responsibility for the evaluation as they progress by using community supports. As community coalition staff become more comfortable with evaluation, the project encourages them to seek evaluation support within their local community. Whether it is a college intern seeking experience or a knowledgeable coalition member, these people are invaluable to the sustainability of the evaluation. As the first five years of the MAP Evaluation Project draw to a close, there have been many valuable lessons learned from the empowerment model of evaluation. These lessons may assist other communities that seek to increase capacity for evaluation and sustain the changes that the programs promote.

Stacy Constantineau Meade
Project Coordinator
Michigan Public Health Institute