Volume X, Number 1, Spring 2004

Issue Topic: Evaluating Out-of-School Time Program Quality

Theory & Practice

Characterizing and Measuring Participation in Out-of-School Time Programs

Honing in on the relationship between breadth of youth activities and outcomes, Sandra Simpkins Chaput from HFRP summarizes recent developmental research examining dimensions of participation in out-of-school activities.

One of the key questions in the out-of-school time (OST) field is, How much program participation is enough to produce beneficial outcomes for participants? However, before this question can be answered, we need to have a deeper understanding of what it means to participate in OST activities. This article synthesizes developmental research and OST evaluations to provide an overview of the dimensions of participation and then hones in on breadth as an important dimension for achieving youth outcomes.¹

Characterizing Participation
Participation, or the time youth spend in OST activities, has several dimensions. To date, most researchers who have examined participation have grouped youth into one of two categories: those who participate in OST activities and those who do not. Although this grouping has been and will continue to be useful in our understanding of OST programs, it overlooks several other indicators of participation. These include:

1. Intensity – the amount of time youth participate in a program during a given period. Intensity has been measured in terms of hours per day, days per week, and weeks per year.
2. Duration – the history of participation. Duration is distinct from intensity. Intensity addresses current participation, whereas duration addresses the number of years youth have participated. For example, two youth may participate with the same intensity (e.g., 3 days per week), but for different durations (e.g., 1 year versus 5 years).
3. Breadth – the variety of participation (Eccles & Barber, 1999). Variety refers to whether participation is focused on one or more types of activities within and/or across programs. For example, many OST programs incorporate breadth by offering children a variety of activities (e.g., reading time, sports, dance). Other programs specialize in one type of activity, such as baseball. In this case, youth can achieve higher breadth by participating in multiple activities across a variety of programs.

Research on Breadth and Outcomes
Breadth is particularly interesting to examine in relation to outcomes because a small but growing research base is indicating that breadth has implications for program quality. However, of the three dimensions of participation, breadth has received the least attention from the research and evaluation community.

Baker and Witt (1996) studied breadth within a multi-component program by examining differences in outcomes depending on the number of activities in which youth participated within the same program. Findings suggest that elementary school children who participated in three or more different activities had higher grades and academic test scores than nonparticipants or youth who participated in only one or two activities. In fact, the outcomes of youth who participated in one or two activities were not significantly different than nonparticipants’ outcomes.

Breadth in activities can also be attained through participation in different programs or activities. Participation in a variety of distinct activities after school is associated with more beneficial outcomes, such as academic achievement and lower drug use (Elder, Leaver-Dunn, Wang, Nagy, & Green, 2000; Gerber, 1996; Swanson, 2002).

Other researchers have studied the number of activities in which youth participate during their high school years, essentially merging breadth and duration. For example, two youth could have each participated in four after school activities during high school. Youth A could have participated in four different activities for just 1 year, thus showing high breadth but low duration. Youth B, on the other hand, could have participated in the same sport for 4 years, thus showing low breadth, but high duration.

Although this characterization makes it impossible to untangle breadth and duration, it has lead to some interesting results. The number of activities in which youth participate during their high school years was positively associated with numerous indicators, including satisfaction with life (Gilman, 2001), academic achievement, homework completion, beliefs about their abilities, educational and occupational plans, and university enrollment (Marsh, 1992; Marsh & Kleitman, 2002). While an indicator combining duration and breadth more accurately reflects the complex nature of participation, it would be helpful if researchers completed additional analyses to understand if the effect is attributed to duration, intensity, breadth, or a combination.

How Is Participation Breadth Associated With Outcomes?
How much participation breadth is necessary to impact child outcomes? Three general models address this question. Several researchers have found a linear relationship between participation breadth and various outcomes (Baker & Witt, 1996; Gerber, 1996; Gilman, 2001; Marsh & Kleitman, 2002). As Figure 1 depicts, the linear model suggests that as participation breadth increases, youth outcomes will increase. According to this model, the more variety in OST activities, the better the outcomes.

However, some of the linear relationships may, in fact, be thresholds. A threshold model, as drawn in Figure 2, suggests that people will benefit if their participation exceeds a certain level or threshold. The findings from Baker and Witt (1996) suggest that there may be a threshold, or a bump in outcomes, when youth participate in three to four activities, and an additional threshold at five or more activities.

While this pattern may hold true for studies that involve middle school students, studies on breadth during adolescence suggest that the relations may be curvilinear (Elder, Leaver-Dunn, Wang, Nagy, & Green, 2000; Marsh, 1992; Marsh & Kleitman, 2002; Swanson, 2002). As shown in Figure 3, moderate amounts of breadth are associated with the best outcomes. High breadth is still associated with better outcomes than no breadth or low breadth.

Figure 1: Linear Model		Figure 2: Threshold Model
Figure 3: Curvilinear Model

Conclusion
Overall, the studies reviewed here suggest that participation in a variety of OST activities, either within a single program, or over the course of the week, is associated with beneficial youth outcomes. Measuring breadth provides unique information that cannot be obtained from indicators that simply distinguish between youth who do and do not participate. For instance, breadth and an indicator of participating/not participating demonstrated beneficial effects of program participation (Baker & Witt, 1996). However, the results on breadth suggest that youth needed to participate in multiple activities within a program to maximize the benefits of participation. This information has important implications for program design and would not have been obtained from only knowing whether children participated or not.

Solely examining the differences between participants versus nonparticipants glosses over many important aspects of participation. While it is true that in many cases simply having a child at the program site may make a difference in terms of safety and supervision, it is time to develop a knowledge base that helps program leaders understand how to promote quality programs that yield maximum impact on their participants. One key ingredient of this effort is to develop more nuanced approaches to measuring and assessing intensity, duration, and, especially, breadth of participation on youth outcomes.

References
*Baker, D., & Witt, P. A. (1996). Evaluation of the impact of two after-school recreation programs. Journal of Park and Recreation Administration, 14, 23–44.

Eccles, J. S., & Barber, B. L. (1999). Student council, volunteering, basketball, or marching band: What kind of extracurricular involvement matters? Journal of Adolescent Research, 14, 10–43.

*Elder, C., Leaver-Dunn, D., Wang, M. Q., Nagy, S., & Green, L. (2000). Organized group activity as a protective factor against adolescent substance use. American Journal of Health Behavior, 24, 108–113.

*Gerber, S. B. (1996). Extracurricular activities and academic achievement. Journal of Research and Development in Education, 30, 42–50.

*Gilman, R. (2001). The relationship between life satisfaction, social interest, and frequency of extracurricular activities among adolescent students. Journal of Youth and Adolescence, 30, 749–767.

*Marsh, H. W. (1992). Extracurricular activities: Beneficial extension of the traditional curriculum or subversion of academic goals? Journal of Educational Psychology, 84, 553–562.

*Marsh, H. W., & Kleitman, S. (2002). Extracurricular school activities: The good, the bad, and the nonlinear. Educational Review, 72, 464–514.

*Swanson, C. B. (2002). Spending time or investing time? Involvement in high school curricular and extracurricular activities as strategic action. Rationality and Society, 14, 431–471.

¹ Eighty methodologically rigorous studies examining the links between participation in general out-of-school activities and outcomes were identified through several searches. These studies are discussed in a forthcoming HFRP Issues and Opportunities in Out-of-School Time Evaluation brief on understanding OST participation. (To be notified when this brief becomes available, sign up for our out-of-school time updates email.) From this group, seven studies included findings relevant to the issues discussed in this article. These seven studies, which cover a range of youth development indicators, OST program and participation indicators, and evaluation and academic research, are marked with an asterisk in the reference list at the end of this article.

Sandra Simpkins Chaput, Research Associate, HFRP