Volume X, Number 4, Winter 2004/2005

Issue Topic: Evaluating Family Involvement Programs

Spotlight

Using Concept Mapping to Develop Family Support Program Theory

Scott Rosas, from the Nemours Foundation, discusses the potential of concept mapping for the design and implementation of family support evaluations.

Although family support has gained widespread acceptance as a viable human-service delivery approach, particular characteristics of family support models often present challenges to determining clear program outcomes. Family support models have emerged from a variety of theoretical and nontheoretical approaches, reflected in the wide array of service mechanisms and diverse perspectives on the scope, content, and effects of programs. The absence of a clear empirical foundation on which to base decisions has led to confusion among developers and evaluators about which aspects of these programs to assess.

Moreover, because family support programs interact uniquely with the local context, they are constantly adjusting to environmental influences, and the same model can vary widely across contexts. Finally, these programs cannot be separated from the numerous individual characteristics and perceptions of those who develop and manage them. Although these issues are not exclusive to family support, they highlight the need for clear conceptual frameworks to guide the design, implementation, and evaluation of family support programs.

Concept Mapping
The need for conceptual clarity has led developers and evaluators to search for ways to develop adequate program theory. One promising approach to theory development is concept mapping.¹ Concept mapping is a multistep process that helps to describe and delineate concepts and their interrelationships through group process, multivariate statistical analyses, and group interpretation. It involves the following four steps:

1. Generation of content through group brainstorming.
2. Organization and prioritization of content through sorting and rating procedures.
3. Analysis and construction of concept maps through the application of multidimensional scaling and hierarchical cluster analysis.²
4. Labeling and interpretation of the maps through stakeholder review.

Family support practitioners, for example, may brainstorm specific benefits for participants, such as peer support, gaining confidence, and communication skills. They then sort each of the brainstormed items into groups based on similarity—social relationships, personal development, resource control—and rate each item on a scale reflecting their relative importance. The analysis produces a concept map that visually represents each item’s place in the hierarchy; for example, relationships may score higher than services. In labeling and interpreting the map, practitioners discuss how it represents their ideas of what happens as a result of participation in the model.

Potential Benefits for Family Support
Concept mapping can assist family support developers and evaluators in articulating a framework of central program concepts. Using concept mapping to guide theory-driven evaluations can help strengthen the conceptual and empirical base of family support. Concept mapping can also help support sound implementation of family support programs. At regular intervals, concept maps could be compared with original program design to monitor the degree to which programs shift or adapt to their environments. Key differences between staff and participant orientations have important implications for program design and evaluation; regular review of concept maps can enable stakeholders to identify disparate views before they lead to implementation problems.

Building and maintaining good stakeholder-evaluator relations is another area in which concept mapping can be of use. Family support evaluations are often collegial and participatory, and require data-gathering techniques to facilitate the socialization of evaluators and agency staff. The application of concept mapping in evaluation can build value for stakeholders, as their input is incorporated throughout the entire process, increasing the likelihood of ongoing engagement in programming and evaluation.

Output from concept mapping can help family support developers and evaluators make informed decisions about design, measurement, analysis, and interpretation. These decisions are crucial; Olds³ found that the inability to specify the benefits families might receive from family support initiatives has been due to an absence of investigations grounded in a clear theoretical framework.

Finally, concept mapping can help facilitate the development of cumulative knowledge about social problem solving. Comparing concept maps across family support programs can highlight common program elements, thereby enabling practitioners and evaluators to more readily identify similarities and differences in application. Thus, concept mapping could be used to identify under what conditions and contexts family support program features contribute to the effectiveness of a particular model.

¹ Trochim, W. M. K. (1989). An introduction to concept mapping for planning and evaluation. Evaluation and Program Planning, 12(1), 1–16.
² Multidimensional scaling is a multivariate statistical technique used to place items spatially on a map according to their relative similarity. Hierarchical cluster analysis is another multivariate statistical technique, used to group points into clusters representing higher order aggregates of the brainstormed items.
³ Olds, D. (1988). Common design and methodological problems encountered in evaluating family support services: Illustrations from the prenatal/early infancy project. In H. B. Weiss & F. H. Jacobs (Eds.), Evaluating family programs (pp. 239–264). New York: Aldine de Gruyter.

Scott Rosas, Ph.D.
Senior Program & Policy Analyst
Nemours Division of Health and Prevention Services
Christiana Building, Suite 200
252 Chapman Road
Newark, DE 19702
Tel: 302-444-9131
Email: srosas@nemours.org