You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.

www.HFRP.org

The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.

Terms of Use ▼


Seattle began funding three demonstration family centers in 1992 through the 1990 City of Seattle Families and Education Levy. Each center receives its primary support from the levy, and obtains additional support through local in-kind contributions, grants, and donations. Based on the results of a three-year evaluation effort and broad-based community support for the demonstration centers, Seattle opened three additional family centers in 1994–1996.

Janet Youngren Miller, Ph.D., conducted the evaluation of Seattle's family centers from 1992–1995. She is a researcher and program evaluator for prevention and family support programs in and around Seattle, and teaches in the family support certificate program at Edmonds Community College. Although the formal evaluation of Seattle's family center is complete, Dr. Miller continues to work with centers on preserving their data collection and evaluation efforts.

The following describes Miller's evaluation approach and the ways in which her strategy met the challenges of family resource center evaluation.

Five-Tiered Approach

The Seattle evaluation was implemented in three one-year phases following Jacob's (1988) five-tiered evaluation approach. This approach organizes evaluation activities at five levels to be performed in order, at the same time, or non-sequentially (see table).

Five-Tiered Evaluation Approach
Tier Evaluation Purpose
Pre-implementation • Document program need
• Demonstrate program fit with community need
• Provide “data groundwork”
Accountability • Document utilization
• Justify expenditures
• Increase expenditures
• Build a constituency
Program Clarification • Provide feedback to staff
Progress Toward Objectives • Provide feedback to staff
• Document effectiveness
Program Impact • Contribute to substantive program or evaluation theory
• Produce evidence of differential program effects
• Suggest models worthy of replication

The evaluation's first phase focused heavily on accountability and program clarification. In conjunction with program staff, the evaluator concentrated on developing capacity for reliable data collection and outcome measures. First year methods included tracking participation rates, observing and videotaping center activities, interviewing staff, surveying center partners and collaborators, and assessing community needs and satisfaction with a participant survey.

During the second phase, the evaluation continued to track center utilization and to solicit feedback from participants on services received. Because increasing social support is a conceptual underpinning of the family center initiative, second year efforts began to focus on this construct as a primary program outcome. The evaluator examined progress toward objectives with a second participant survey and focus groups examining evidence of social support effects.

Further Reading


Jacobs, F. (1988). The five-tiered approach to evaluation: Context and implementation. In H. Weiss & F. Jacobs (Eds.), Evaluating family programs. New York: Aldine De Gruyter.

Weiss, C. H. (1995). Nothing as practical as good theory: Exploring theory-based evaluation for comprehensive community initiatives for children and families. In J. Connell, A. Kubisch, L. Schorr, & C. Weiss (Eds.), New approaches to evaluating community initiatives. Washington, DC: The Aspen Institute.

The third phase focused on documenting program outcomes and impact, treating participation rates as valid indicators of program value, and relying heavily on direct participant feedback for evidence of program impact. The evaluation incorporated a fifth tier strategy through the analysis of participant survey data for evidence of differential program effects. Third year methods again included utilization tracking, meeting and activity participation and observation, surveys of participants and partners, focus groups, and interviews with staff and participants.

Using a Five-Tiered Approach to Meet Evaluation Challenges

Miller's five-tiered approach follows an inductive exploratory methodology that allows for flexibility and center responsiveness. She adjusts the evaluation to fit and meet center needs and activities, initially focusing heavily on the second and third tiers, striving to develop data collection capacity and provide feedback to staff for program clarification. After the program developed clear service delivery and data collection processes, the evaluation shifted from accountability and clarification tiers to documenting program outcomes and impact.


Julia Coffman
HFRP Research Specialist

For more information about the evaluation, please contact:

Janet Miller Evaluation Services
5599 Crystal Springs Road
Bainbridge Island, WA 98110
Tel: 206-780-2513

‹ Previous Article | Table of Contents | Next Article ›

© 2016 Presidents and Fellows of Harvard College
Published by Harvard Family Research Project