You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.

www.HFRP.org

The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.

Terms of Use ▼

Research Description

Overview and Components The Study of Promising After-School Programs aims to (a) identify after school programs that feature promising practices for enhancing youth’s academic and social development and their emotional and physical well-being and (b) test the hypothesis that disadvantaged youth who participate in promising after school programs achieve greater learning and developmental gains over a 2-year period than disadvantaged nonparticipants. Promising after school programs are those that offer high-quality environments for youth, are sustainable, and promote positive youth outcomes.
Start Date The study began in the fall of 2002. (completed in spring 2005).
Scope national
Type after school
Location urban, suburban, rural
Setting public schools, community-based organizations
Participants elementary and middle school students (ages 8–14)
Number of Sites/Grantees 37 in 2002–2003; 35 in 2003–2004; 34 in 2004–2005
Number Served average enrollment of 328 youth for elementary school programs and 504 youth for middle school programs in 2002–2003
Study Details Year 1 of the study identified promising programs. Year 2 examined intermediate impacts of promising programs and other after school experiences on youth outcomes. Year 3 examined longer terms impacts of programs and other after school experiences on youth outcomes.

Selection criteria for promising programs stipulated that programs be school linked or school based; serve elementary or middle school youth from low-income families; meet at least 3 days a week; charge families no fee or only a small fee; expect to be sustained for the next 3 years; have been in operation for at least 3 years; offer opportunities for sustained youth involvement in substantive activities that support skill building, mastery, and engagement; have access to resources and materials to support substantive activities; use staffing patterns that contribute to positive and supportive youth relationships with staff and peers (low child–adult ratios and staff turnover, staff with training and expertise); serve at least 30 youth in target grades (grades 3–4 and 6–7); and show evidence of positive youth impacts in a previous evaluation.
Funding Level Not available
Funding Sources The C. S. Mott Foundation funded the study. Government agencies, private/corporate donors, school districts, and private foundations funded the programs.
Researchers Wisconsin Center for Education Research and Policy Studies Associates
Research Profiled Descriptive Report of the Promising Programs.

Examination of Intermediate Outcomes in Year 2

Examination of Longer Term Outcomes After Two Years of Program Experience
Research Planned None
Report Availability Vandell, D. L., Reisner, E. R., Brown, B. B., Pierce, K., Dadisman, K., & Pechman, E. M. (2004). The study of promising after-school programs: Descriptive report of the promising programs. Madison: Wisconsin Center for Education Research.

Vandell, D. L., Reisner, E. R., Brown, B. B., Dadisman, K., Pierce, K. M., Lee, D., & Pechman, E. M. (2005). The study of promising after-school programs: Examination of intermediate outcomes in year 2. Madison: Wisconsin Center for Education Research.

Vandell, D. L., Reisner, E. R., Pierce, K. M., Brown, B. B., Lee, D., Bolt, D., & Pechman, E. M. (2006). The study of promising after-school programs: Examination of longer term outcomes after two years of program experiences. Madison: Wisconsin Center for Education Research.

Available at www.wcer.wisc.edu/childcare/statements.html


Contacts

Research Deborah Lowe Vandell
Chair, Department of Education
University of California, Irvine
2001 Berkeley Place
Irvine, CA 92697-5500
Tel: 949-824-7840
Fax: 949-824-2965
Email: dvandell@uci.edu
Profile Updated November 17, 2006

Research Study 1: Descriptive Report of the Promising Programs


Research Description

Research Purpose To examine the quality and implementation of potentially promising after school programs.
Research Design Non-Experimental: Based on a review of published materials, recommendations from after school experts (academics, practitioners, youth-serving organizations, consultants, and federal and state officials), and interviews with program directors, 35 elementary and 26 middle school programs met the study criteria and thus were identified as potential study sites. Considerations of geographic diversity and accessibility to the researchers led to the selection of 29 elementary and 28 middle school sites for fall 2002 site visits. During these visits, activities were observed over 2 days and interviews were conducted with program directors, activity leaders (one at each site), and principals at all host schools. In spring 2003, researchers returned to 19 elementary and 18 middle school programs that seemed especially promising based on fall visits, during which researchers observed activities again over 2 days and conducted program director and activity leader surveys. Of the sites visited in the spring, 13 were in California, 11 in the Northeast, 9 in the Northwest, and 4 in the Midwest; about 40% (slightly more at the middle school than elementary level) were in major metropolitan centers, another 40% in other urban areas, and the remaining 15–20% in smaller towns or rural areas. On average, elementary programs had been in operation for over 5 years and middle school programs for 8 years.
Data Collection Methods Interviews/Focus Groups: Director and activity leader interviews asked about program activities/schedules, youth opportunities for autonomy and choice, and staff training. Directors were also asked about program goals and funding, relationships with community agencies, youth enrollment/attendance, and staff education. Activity leaders were asked about planning opportunities and relationships with partner schools. Principal interviews asked about school enrollment/demographics and perceptions of the program.

Observation: Observations collected data about program quality and implementation.

Surveys/Questionnaires: All surveys asked about staff training/background and partner school–parent relations. Directors were asked about staff meetings, program enrollment, attendance, space/material resources, funding, and community agency relations. Activity leaders were asked about their experience, planning opportunities, and job satisfaction/support.

Test/Assessments: Two observations tools were used: (a) the Promising Practices Rating System (PPRS, Vandell et al., 2004), developed for this study, was used during fall site visits to quantify 7 program processes related to quality, including supportive relations with adults; supportive relations with peers; youth engagement; appropriate program structure; cognitive growth opportunities; mastery orientation (e.g., “activities involve a graded progression of skills”); and autonomy opportunities; and (b) the After-School Activity Observation Instrument (AOI, Pechman & Marzke, 2003), an observation/interview measure modified for spring site visits to collect data during 15-minute observations of content and skill areas, use of space/materials, youth interactions with peers/staff, youth engagement, and activity structure. The AOI identifies 21 types of activities (e.g., arts and crafts, snack, etc.), of which all but two were observed in programs. To get a sense of how well each type of activity reflected the PPRS processes, means scores on scales related to each of these features was calculated (all except autonomy opportunities), and activities types in which the mean score for a given scale were especially high were identified.

References:
Pechman, E., & Marzke, C. (2003). After-School Activity Observation Instrument (AOI) observation manual. Adapted for the WCER/PSA Study of Promising After-School Programs. Washington, DC: Policy Studies Associates.

Vandell, D. L., Reisner, E. R., Brown, B. B., Pierce, K., Dadisman, K., & Pechman, E. M. (2004). The study of promising after-school programs: Descriptive report of the promising programs. Madison: Wisconsin Center for Education Research.
Data Collection Timeframe Data were collected in fall 2002 and spring 2003.


Findings:
Formative/Process Findings

Activity Implementation The majority of programs (71% of elementary and 78% of middle school programs) operated at least 5 days a week. On average, programs were open just under 4 hours a day.

Observations revealed that all programs offered academic assistance (e.g., homework help, tutoring, study skills, test prep). Board games/puzzles, arts and crafts, and performing arts activities were also common. Computer-oriented activities were more common in middle school programs, while elementary programs were more likely to have reading/language arts enrichment. Program staff reported a variety of other activities that were not observed, including photography, youth-run businesses, video production, outdoor skills, drama, textile production, cooking, gardening, oral history, drill team, and community service.

Program schedules reflected community needs and the constraints imposed by location, resources, staffing, and funding. Marked schedule differences were sometimes found from one day or season to the next. Most programs had multiple classes going simultaneously at various points in the year. Activities tended to change regularly; the majority of directors noted that they offered activities in distinct sessions. Most elementary sessions ran 7–12 weeks, while nearly half of middle school sessions ran 1–6 weeks. The number of days an activity was offered per week was inconsistent, but for the majority of elementary programs, daily activity sessions lasted an hour or less, while middle school programs tended to allow 1–2 hours.

Average program quality ratings ranged from 3.0 (somewhat characteristic of the program) to just below 4.0 (highly characteristic of the program). Average ratings were highest for supportive relationships with peers and adults and for youth engagement. Middle school programs outscored elementary programs on autonomy opportunities and emphasis on mastery. Cognitive growth opportunities were comparatively limited, especially in middle school programs.

Nearly three quarters of elementary and almost all middle school programs received average quality scores of at least 3.5 out of 4 on more than half of the 7 processes measured, and a fifth of elementary and half of middle school programs had such ratings on at least 6 processes. Over half of programs received 4s for youth engagement and supportive peer relationships. Less than 20% of programs received top scores for enhancing cognitive growth. Top ratings were more prevalent among middle school programs for all processes except appropriate structure.

Most directors and activity leaders reported youth leadership opportunities, which most commonly arose without formal organization. In some programs, staff systematically encouraged older youth or youth with more advanced skills to take responsibility for other youth. Some programs had high school youth on staff to help with activities.

High ratings by activity type covered all program process areas except appropriate structure and each process area was facilitated by at least two activity types. No activities type received high ratings in all 5 of the remaining areas, but a number of activities received high rating in 4 of the 5 areas: at the elementary level in performing arts and snacks and at the middle school level in reading/language arts enrichment, tutoring, computer skill building, higher education/ career prep, and study skills/test prep.
Costs/Revenues Over two thirds of programs drew funds from local/federal government agencies; about half from private/corporate donors, over 40% from local school districts, and a third from private foundations. Many also had national foundation grants (33% of elementary and 43% of middle school programs). In terms of dollar amounts, the federal government accounted for over a third of elementary and 45% of middle school program budgets, while local governments accounted for a fourth of elementary and nearly a fifth of middle school program budgets. Much of the rest of elementary budgets came from private/corporate donors. The last third of middle school budgets came more equally from several sources.
Parent/Community Involvement Community organizations (schools, businesses, and social service agencies) provided youth activities, volunteers, and supplies at almost all programs. Many sites also counted on these partners to provide mentors, classes for parents, funding, youth referrals, and activity space. Programs relied on multiple organizations for resources; many reported associations with 5 or more organizations for any given resource. In general, elementary programs reported more community linkages than did middle school programs.

Directors sent information home and held events for parents at least once a semester. For elementary directors, the majority spoke to (90%) and met with (80%) parents several times a month. In addition, many programs offered classes for parents, dealing with childrearing, employment skills, health care, and English language learning.
Program–School Linkages Over two thirds of elementary program directors spoke to school staff “almost weekly” about homework and individual youth’s needs. The majority of directors spoke to school staff about curriculum concepts at least a couple of times per semester. Over half of directors met with the principal at least 2–3 times a month; over a third did so weekly. Such meetings were more frequent in elementary programs.

Over a third of activity leaders spoke to school staff about individual youth almost weekly. Nearly half of elementary leaders spoke to school staff about homework almost weekly. Communication about curriculum concepts was less frequent, especially among elementary leaders. About 40% of leaders spoke to school staff about classroom space at least occasionally.
Program Context/Infrastructure Almost all programs had classroom, storage, staff planning, and meeting space; at least three quarters had access to outdoor space, a cafeteria, and specialty areas (gym, library, or art room); and a majority also had auditorium, computer lab, and kitchen access. The only area that was less common was science labs for elementary programs. Storage and library space, however, were judged to be inadequate by over a third of directors, and many elementary directors found art, music, and office space to be limited.

Over 80% of directors had access to all types of materials listed on the survey, and very few reported limited materials of any particular type. Almost all directors, however, felt that games and puzzles were in short supply, almost 1 in 3 elementary directors felt that computer access was inadequate, and 1 in 4 middle school directors noted a need for more physical education equipment and expanded transportation services for field trips.

Across the schools associated with the programs, 83% of elementary youth and 75% of middle school youth qualified for free/reduced-price lunch.
Recruitment/Participation Directors reported that 65% of elementary and 50% of middle school youth attended programs 3 or more days a week. Three quarters of elementary programs indicated that at least half of their participants attended 3 or more days a week, and 1 out of 5 claimed this benchmark was achieved by more than 90% of their participants. By contrast, only 57% of middle school programs reported that a majority of their youth attended 3 or more days a week, and none indicated that 90% or more of their youth achieved this benchmark.

Latinos made up the highest percentage of participants (56% elementary, 43% middle school), followed by White (23% elementary, 33% middle school), Black (18% elementary, 16% middle school), and Asian and other (1–5%) youth. Participants closely matched the ethnic compositions of the associated schools.
Staffing/Training Activity leaders’ expressed satisfaction overall with working conditions and staff relations.

Elementary programs reported 6–28 staff (average = 15) and middle school programs reported 5–44 (average = 16), including paid and volunteer staff in a variety of roles, from administrators to activity leaders to custodians. An average youth–staff ratio, including teen volunteers, was observed to be 10:1 in elementary and 8:1 in middle school programs.

On average, directors had been at programs for 3.6 years and worked just under 5 days a week. They worked for 29 hours a week at elementary sites and 38 hours at middle school sites. All were paid by the program; many also worked for the school (38% at elementary and 47% at middle school sites). The majority held a bachelor’s degree or higher (84% of middle school and 69% of elementary directors). Almost 40% of middle school and 7% of elementary directors were certified teachers. Most directors (80%) had previously worked in youth recreation/childcare positions; about 3 of 4 had classroom instruction experience; and over a third had administrative experience in youth and/or social service work.

On average, activity leaders worked 10–12 hours across a little over 3 days a week. They had been at the program for an average of 2.2 years at elementary sites and 1.6 years at middle school sites. About 80% were paid by the program; about half of middle school and a fourth of elementary leaders also worked for the school. About half held a bachelor’s degree or higher. Nearly a third of middle school and a fifth of elementary leaders were certified teachers. The majority had prior experience in youth and/or social services or classroom instruction.

Few directors reported hiring teen staff. At elementary sites, teens worked an average of 15 hours over 4 days a week and had 1.5 years of experience; all were paid by the program. At middle school sites, teens worked an average of 3 hours over 2 days a week and had an average of a few months of experience; none were paid by the program.

Other staff worked, on average, 3–4 days for 10–12 hours per week and had been at the program for 2 years. The majority were paid by the program (83% at elementary and 69% at middle school sites), and some also worked for the school (15% and 40% respectively).

Staff training was provided in a variety of areas, but most commonly in classroom management and academic enrichment. For middle school programs, staff training was also commonly provided in conflict resolution, diversity training, and youth development. In elementary programs, staff were less likely to receive arts or diversity training than in middle school programs, and training was more likely to be offered to paid staff than to volunteers. On average, directors reported receiving 51 hours of training in the last year; activity leaders reported 21 hours.

Most activity leaders reported that they held major responsibilities for planning activities, which occurred at least weekly in nearly 80% of cases. About a quarter of elementary and over a third of middle school program activity leaders said that they engaged in activity planning on a daily basis. Nearly all leaders met with other staff to discuss program issues; a third of elementary and close to half of middle school leaders reported that such meetings occurred weekly.

© 2016 Presidents and Fellows of Harvard College
Published by Harvard Family Research Project