Jump to:Page Content
You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.
The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.
Volume ll, Number 2, Fall 1996
Issue Topic: Family Resource Centers
Evaluations to Watch
Janet Youngren Miller, Ph.D., conducted the evaluation of Seattle's family centers from 1992–1995. She is a researcher and program evaluator for prevention and family support programs in and around Seattle, and teaches in the family support certificate program at Edmonds Community College. Although the formal evaluation of Seattle's family center is complete, Dr. Miller continues to work with centers on preserving their data collection and evaluation efforts.
The following describes Miller's evaluation approach and the ways in which her strategy met the challenges of family resource center evaluation.
Five-Tiered Approach
The Seattle evaluation was implemented in three one-year phases following Jacob's (1988) five-tiered evaluation approach. This approach organizes evaluation activities at five levels to be performed in order, at the same time, or non-sequentially (see table).
|
|
Tier | Evaluation Purpose |
Pre-implementation | • Document program need • Demonstrate program fit with community need • Provide “data groundwork” |
Accountability | • Document utilization • Justify expenditures • Increase expenditures • Build a constituency |
Program Clarification | • Provide feedback to staff |
Progress Toward Objectives | • Provide feedback to staff • Document effectiveness |
Program Impact | • Contribute to substantive program or evaluation theory • Produce evidence of differential program effects • Suggest models worthy of replication |
The evaluation's first phase focused heavily on accountability and program clarification. In conjunction with program staff, the evaluator concentrated on developing capacity for reliable data collection and outcome measures. First year methods included tracking participation rates, observing and videotaping center activities, interviewing staff, surveying center partners and collaborators, and assessing community needs and satisfaction with a participant survey.
During the second phase, the evaluation continued to track center utilization and to solicit feedback from participants on services received. Because increasing social support is a conceptual underpinning of the family center initiative, second year efforts began to focus on this construct as a primary program outcome. The evaluator examined progress toward objectives with a second participant survey and focus groups examining evidence of social support effects.
Further ReadingJacobs, F. (1988). The five-tiered approach to evaluation: Context and implementation. In H. Weiss & F. Jacobs (Eds.), Evaluating family programs. New York: Aldine De Gruyter. Weiss, C. H. (1995). Nothing as practical as good theory: Exploring theory-based evaluation for comprehensive community initiatives for children and families. In J. Connell, A. Kubisch, L. Schorr, & C. Weiss (Eds.), New approaches to evaluating community initiatives. Washington, DC: The Aspen Institute. |
The third phase focused on documenting program outcomes and impact, treating participation rates as valid indicators of program value, and relying heavily on direct participant feedback for evidence of program impact. The evaluation incorporated a fifth tier strategy through the analysis of participant survey data for evidence of differential program effects. Third year methods again included utilization tracking, meeting and activity participation and observation, surveys of participants and partners, focus groups, and interviews with staff and participants.
Using a Five-Tiered Approach to Meet Evaluation Challenges
Miller's five-tiered approach follows an inductive exploratory methodology that allows for flexibility and center responsiveness. She adjusts the evaluation to fit and meet center needs and activities, initially focusing heavily on the second and third tiers, striving to develop data collection capacity and provide feedback to staff for program clarification. After the program developed clear service delivery and data collection processes, the evaluation shifted from accountability and clarification tiers to documenting program outcomes and impact.Julia Coffman
HFRP Research Specialist
For more information about the evaluation, please contact:
Janet Miller Evaluation Services
5599 Crystal Springs Road
Bainbridge Island, WA 98110
Tel: 206-780-2513