Volume X, Number 4, Winter 2004/2005

Issue Topic: Evaluating Family Involvement Programs

Beyond Basic Training

What Matters in Family Support Evaluation?

M. Elena Lopez of HFRP interprets themes from a participatory evaluation and parent engagement institute.

What matters in family support evaluation? In September 2004, researchers, evaluators, practitioners, and parent leaders gathered at the Ewing Marion Kauffman Conference Center in Kansas City, Missouri, to examine evaluation approaches that meet the information needs of various stakeholders and uphold family support principles. Convened by Family Support America, the Participatory Evaluation and Parent Engagement Institute created a forum to discuss the strengths, challenges, and new directions of family support evaluations.

The Institute highlighted programs that conduct family support evaluations in alignment with family support principles. Participatory evaluation in particular reflects several family support principles, among them building the skills of families and forming relationships based on equality and respect. Institute presenters, which included parent leaders, staff members, and evaluators, shared lessons from parents’ involvement in evaluation:

• When adequately trained and supported, parents can be involved in all phases of an evaluation, from planning to using evaluation findings.
• Parents bring diverse perspectives, which contribute to balanced views on issues and to outcomes and indicators relevant to families and their contexts.
• Parents and community members raise important questions about data ownership and outcomes, questions that spark deeper discussions about race, class, and power.
• Parents and evaluators are co-learners: Parents can help evaluators learn to communicate clearly to a lay audience an evaluation’s purpose, methods, and findings; evaluators can train parents to enhance their critical thinking skills.

Although family support has grown over the last three decades, it is fragile. Programs tend to be underfunded and to lack capacity to deliver quality services. Evaluation can help family support achieve its vision of a high quality, sustainable system of care for children, youth, and families. Although no one evaluation methodology fulfills this function, several evaluation approaches can build a strong case for the direction and strategy of the field—particularly urgent, as the field operates in a policy environment of competing social programs and scarce resources. In addition, family support evaluation must meet stringent criteria of -evidence-based practice and build a learning system that connects evaluation stakeholders at different decision-making levels.

To tackle these challenges, Heather Weiss, of Harvard Family Research Project, urged Institute participants to implement a comprehensive evaluation strategy. As the Institute’s final keynote, this strategy connected several strands of the various presentations and included four components:

1. Experimental evaluations with randomized assignments can demonstrate the value-added of family support practices and whether their implementation leads to improved outcomes for families. Although this type of evaluation carries its own design, implementation, and financial challenges, family support evaluators from different countries are coming to the conclusion that failure to conduct experimental evaluations poses a greater risk for replication and sustainability than conducting evaluations that report negative findings.
2. Utilization-focused evaluation can help practitioners use the results to make decisions about training and program development, as well as to demonstrate accountability. The Office of Child Development at the University of Pittsburgh, for example, presented its family support management information system, which allows centers to use data for program management, improvement, and outcome evaluation. The system uses a set of outcomes agreed upon by family support centers and their parent councils.
3. Action research and participatory evaluation enable families and program practitioners to use data for community change. Denver’s Community Learning Network¹ presented a resident-driven evaluation process. The Network encourages residents to take the lead in defining their research issues and equips them with the skills to understand, use, and generate data to solve problems. It has evaluated three school-based literacy models and supported a resident-conducted survey of neighborhood crime.
4. The development of performance standards in family support practice—based on research and evaluation—supports a system of high quality care for families. Recent work presented by the Orelena Hawks Puckett Institute on program adherence to family support principles and its impact on parenting behavior illustrates the advances in this direction. Adherence to family support principles can be used as a measure of program quality.

Drawing on the wisdom of over two decades of practice, the family support field must now position itself to act on the demands for research-based programs and evidence about the benefits of family support.

¹ The principles that guide the research of Denver’s Community Learning Network can be found at www.aecf.org/initiatives/ldu/strategies/denver_principles.htm.

M. Elena Lopez, Senior Consultant, HFRP