You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.

www.HFRP.org

The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.

Terms of Use ▼

Research Description

Overview and Components The Study of Promising After-School Programs aims to (a) identify after school programs that feature promising practices for enhancing youth’s academic and social development and their emotional and physical well-being and (b) test the hypothesis that disadvantaged youth who participate in promising after school programs achieve greater learning and developmental gains over a 2-year period than disadvantaged nonparticipants. Promising after school programs are those that offer high-quality environments for youth, are sustainable, and promote positive youth outcomes.
Start Date The study began in the fall of 2002. (completed in spring 2005).
Scope national
Type after school
Location urban, suburban, rural
Setting public schools, community-based organizations
Participants elementary and middle school students (ages 8–14)
Number of Sites/Grantees 37 in 2002–2003; 35 in 2003–2004; 34 in 2004–2005
Number Served average enrollment of 328 youth for elementary school programs and 504 youth for middle school programs in 2002–2003
Study Details Year 1 of the study identified promising programs. Year 2 examined intermediate impacts of promising programs and other after school experiences on youth outcomes. Year 3 examined longer terms impacts of programs and other after school experiences on youth outcomes.

Selection criteria for promising programs stipulated that programs be school linked or school based; serve elementary or middle school youth from low-income families; meet at least 3 days a week; charge families no fee or only a small fee; expect to be sustained for the next 3 years; have been in operation for at least 3 years; offer opportunities for sustained youth involvement in substantive activities that support skill building, mastery, and engagement; have access to resources and materials to support substantive activities; use staffing patterns that contribute to positive and supportive youth relationships with staff and peers (low child–adult ratios and staff turnover, staff with training and expertise); serve at least 30 youth in target grades (grades 3–4 and 6–7); and show evidence of positive youth impacts in a previous evaluation.
Funding Level Not available
Funding Sources The C. S. Mott Foundation funded the study. Government agencies, private/corporate donors, school districts, and private foundations funded the programs.
Researchers Wisconsin Center for Education Research and Policy Studies Associates
Research Profiled Descriptive Report of the Promising Programs.

Examination of Intermediate Outcomes in Year 2

Examination of Longer Term Outcomes After Two Years of Program Experience
Research Planned None
Report Availability Vandell, D. L., Reisner, E. R., Brown, B. B., Pierce, K., Dadisman, K., & Pechman, E. M. (2004). The study of promising after-school programs: Descriptive report of the promising programs. Madison: Wisconsin Center for Education Research.

Vandell, D. L., Reisner, E. R., Brown, B. B., Dadisman, K., Pierce, K. M., Lee, D., & Pechman, E. M. (2005). The study of promising after-school programs: Examination of intermediate outcomes in year 2. Madison: Wisconsin Center for Education Research.

Vandell, D. L., Reisner, E. R., Pierce, K. M., Brown, B. B., Lee, D., Bolt, D., & Pechman, E. M. (2006). The study of promising after-school programs: Examination of longer term outcomes after two years of program experiences. Madison: Wisconsin Center for Education Research.

Available at www.wcer.wisc.edu/childcare/statements.html


Contacts

Research Deborah Lowe Vandell
Chair, Department of Education
University of California, Irvine
2001 Berkeley Place
Irvine, CA 92697-5500
Tel: 949-824-7840
Fax: 949-824-2965
Email: dvandell@uci.edu
Profile Updated November 17, 2006

Research Study 3: Examination of Longer Term Outcomes after Two Years of Program Experience


Research Description

Research Purpose To examine program impacts of sets of experiences in different after school contexts across years 2 and 3.
Research Design Quasi-Experimental: Two middle school programs in year 2 closed; of these, 1 was transferred to a new school, and thus included in the study for a total of 19 elementary and 15 middle school programs in year 3 (2003–2004). Programs maintained their ratings on all program quality indicators from year 2 to 3.

In year 3, 80% of the year 2 elementary sample (n = 1,434) and 76% of the middle school sample (n = 855) remained at the participating schools and were available for data collection. Youth who dropped from the study were significantly less likely to attend the programs during year 2 (p < .001), and more likely to be in the supervised at home cluster in the analysis (p < .01). The elementary youth who left the study were also significantly more likely to be White and less likely to be Hispanic than those who stayed in the study (p < .05).

The program sample was defined as those who attended a program for at least 5 days in at least one semester. In year 3, 57% of the remaining elementary sample and 42% of the remaining middle school sample attended a program at least 1 day, with average attendance rates of 86 and 62 days, respectively. For years 2 and 3 combined, 64% of the elementary sample and 59% of the middle school sample attended a program at least 1 day, with average 2 year attendance rates of 141 and 79 days, respectively. Over the 2 years, the largest proportion of the sample had minimal program dosage (0–9 days; 39% elementary, 49% middle school), with smaller percentages classified as low (10–59 days; 17%, 23% respectively), moderate (60–119 days; 12%, 14%), substantial (120–179 days; 11%, 7%), and high (180 or more days; 21%, 7%) dosage. Over the 2 years, youth participated in other organized after school activities an average of about once a week or less, with youth engaging in lessons most often, and caring for siblings least often. Correlations of the frequency of these activities over 3 time periods (fall 2003, spring 2004, and spring 2005) were small to moderate, suggesting that participation in these activities was somewhat fluid across a single year and across years.

Classroom teacher, youth, and parent surveys were collected and programs were observed over 2 days in spring 2005. Teacher surveys were collected from the regular classroom elementary school teachers and language arts middle school teachers on study sample youth. Of the elementary sample, 70% of youth completed surveys, teacher surveys were completed for 68% of youth, and parent surveys were completed for 56% of youth. Of the middle school sample, 68% of youth completed surveys; teacher surveys were received for 63% of youth; and parent surveys were received for 43% of youth.

Cluster analysis (a statistical technique to group individuals based on scores on a set of variables) was used to create youth clusters based on 8 variables across the 2 years: program status (program vs. comparison group); program dosage (amount of program participation); and the amount of time spent in 6 other after school contexts (school-based extracurricular activities, coached sports, lessons, home alone, home with siblings but no adults present, and hanging out with peers unsupervised by adults). Four clusters were identified: (a) program + activities—high involvement levels in both programs and other enrichment activities (18% of elementary and 17% of middle school youth); (b) program only—high program participation rates and low scores on most of the other variables (37% of elementary and 32% of the middle school youth); (c) self care + activities—low program participation levels, moderate amounts of time in other activities (mostly coached sports), and high amounts of unsupervised time, especially hanging out with peers (15% of both elementary and middle school youth); and (d) supervised at home–relatively low scores on all variables (30% of elementary and 36% of middle school youth). The self care + activities cluster was contrasted with the other clusters to examine whether the programs and other enrichment activities were protective for youth at risk socially and academically.

Clusters differed significantly on a number of demographic variables. In the elementary sample, the program + activities group had the largest proportion of girls, whereas boys predominated in the self-care + activities cluster (p < .001. The program + activities and supervised at home groups had the highest proportions of Hispanic youth (p < .001). The supervised at home group contained relatively more youth from two-parent households and youth whose mothers did not work full-time (p < .001). The self-care + activities group had the highest proportion of mothers who did not graduate from high school (p < .05). Family incomes did not differ between elementary clusters. In the middle school sample, the supervised at home group had the largest proportion of girls (p < .05) and youth from two-parent households (p < .01) as well as the smallest proportion of mothers who worked full-time (p < .01). The self-care + activities cluster had the highest percentage of mothers who worked full time (p < .01), and the supervised at home and program only groups had lower incomes than the other groups (p < .05). The program + activities and program only clusters were predominantly Hispanic and had fewer proportions of White students (p < .05). Maternal education did not vary significantly between the middle school clusters. These differences were controlled for in the analyses.
Data Collection Methods Observation: Programs were observed to confirm that they met the study’s quality criteria.

Secondary Source/Data Review: Program attendance data were collected and used to identify 5 dosage levels based on the number of days youth attended across years 2 and 3.

Surveys/Questionnaires: Surveys included measures of youth’s work habits (all surveys); task persistence/self-efficacy (teachers and middle school youth surveys); social skills, aggressive/prosocial behavior with peers, and academic performance (teacher surveys); substance use (middle school youth surveys); misconduct and time spent in other after school activities (youth surveys); and peer reliance and compliance (parent surveys).

Test/Assessments: A modified version of the Promising Practices Rating System (PPRS, Vandell et al., 2004), developed for this study, was used during observations. It included 8 program processes related to quality: supportive relations with adults; supportive relations with peers; youth engagement in program activities; appropriate program structure; opportunities for cognitive growth; mastery orientation (e.g., “activities involve a graded progression of skills”); chaos in the program; and over-control of youth by staff.

Surveys incorporated items from assessment scales (often modified for this study) to examine youth outcomes, including: (a) the Mock Report Card’s (Pierce et al., 1999) Work Habits Scale, a measure of classroom work habits (youth and teacher surveys), and Academic Performance scale, a measure reading and oral/written language performance (all teacher surveys; elementary teachers also rated math, science, and social studies); (b) the Self-Reported Behavior Index’s Misconduct Scale (Brown, Clasen, & Eicher, 1986), a measure of behavior (youth surveys), and items that ask about frequency of tobacco, alcohol, marijuana, and other drug use (middle school youth surveys); (c) Walker & Arbreton’s Self-Efficacy Scale (2001) a measure of task persistence/self efficacy; (middle school youth and teachers); (d) the Teacher Checklist of Peer Relations’ Prosocial Behavior Scale (Coie & Dodge, 1988), a measure of social skills with peers (teacher surveys); (e) the Child Behavior Scale (Ladd & Profilet, 1996), a measure of aggressive, withdrawn, and prosocial behaviors, with items forming 2 scales, Aggressive with Peers and Prosocial with Peers (teacher surveys); and (f) the Child Adjustment Scale (Santrock & Warshak, 1979), a measure of socioemotional adjustment, which includes scales of work habits, peer relations, and compliance (parent surveys).

References:
Brown, B. B., Clasen, D. R., & Eicher, S. A. (1986). Perceptions of peer pressure, peer conformity, dispositions, and self-reported behavior among adolescents. Developmental Psychology, 22, 521–530.

Coie, J. D., & Dodge, K. A. (1988). Multiple sources of data on social behavior and social status in the school: A cross-age comparison. Child Development, 59, 815–829.

Ladd, G. W., & Profilet, S. M. (1996). The Child Behavior Scale: A teacher-report measure of young children’s aggressive, withdrawn, and prosocial behaviors. Developmental Psychology, 32(6), 1008–1024.

Pierce, K. M., Hamm, J. V., & Vandell, D. L. (1999). Experiences in after-school programs and children’s adjustment in first-grade class classrooms. Child Development, 70, 756–767.

Santrock, J. W., & Warshak, R. A. (1979). Father custody and social development in boys and girls. Journal of Social Issues, 35, 112–125.

Vandell, D. L., Reisner, E. R., Brown, B. B., Pierce, K., Dadisman, K., & Pechman, E. M. (2004). The study of promising after-school programs: Descriptive report of the promising programs. Madison: Wisconsin Center for Education Research.

Walker, K. E., & Arbreton, A. J. A. (2001). Working together to build Beacon Centers in San Francisco. Philadelphia: Public/Private Ventures.
Data Collection Timeframe Data were collected in spring 2005 (year 3) and compared to data collected during the 2003–2004 school year (year 2).


Findings:
Summative/Outcome Findings

Academic Compared to the self-care + activities cluster, significantly greater gains were seen over the 2 years at the elementary level in the other three groups’ youth- and teacher-reported work habits (p < .01 for the program only group’s youth reports and the supervised at home group’s teacher reports, p < .001 for the rest), teacher-reported task persistence (p < .001 for the programs + activities cluster, p < .01 for the other two); the program + activities (p < .01) and the supervised at home (p < .001) groups’ teacher-reported academic performance, and the supervised at home group’s parent-reported work habits (p < .05). At the middle school level, significantly greater gains were seen in the two program groups’ youth-reported work habits (p < .05) compared to the self-care + activities cluster.
Prevention Compared to the self-care + activities cluster, significantly greater decreases were seen over the 2 years for all youth-reported misconduct (p < .001); elementary-teacher-reported aggression with peers (p < .05 for the programs + activities group, p < .01 for the program only group, and p < .001 for the supervised at home group). All three middle school groups’ youth-reported substance abuse declined more than the self-care + activities group (p < .01 for the supervised at home group and p < .001 for the other two).
Youth Development Compared to the self-care + activities cluster, significantly greater gains were seen over the 2 years at the elementary and middle school level for the program only group’s (p < .001) and supervised at home group’s (p < .01) parent-reported compliance; At the elementary level, all three other groups’ teacher-reported social skills (p < .05 for the program only group and p < .01 for the other two) and prosocial behavior (p < .01 for the supervised at home group and p < .05 for the other two) improved relative to the self-care + activities group.

© 2016 Presidents and Fellows of Harvard College
Published by Harvard Family Research Project