You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.

www.HFRP.org

The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.

Terms of Use ▼

Program Description

Overview The Extended-Service Schools Initiative (ESS) supports organizations in 17 cities to provide high quality youth development activities in school buildings during nonschool hours. Services are provided through partnerships between a local low-income public school district and a community-based organization (CBO) and/or a university. The 17 cities are: Oakland and Long Beach, California; Savannah and Atlanta, Georgia; Denver and Aurora, Colorado; Guilford County and Highpoint, North Carolina; Albuquerque, New Mexico; Salt Lake City, Utah; Central Falls, Rhode Island; Minneapolis, Minnesota; Mesa, Arizona; Boston, Massachusetts; Missoula, Montana; Flint, Michigan; and Jacksonville, Florida.
Start Date 1998
Scope national
Type after school, summer/vacation, before school, weekend, comprehensive services
Location urban
Setting public schools
Participants kindergarten though high school students, other community members
Number of Sites/Grantees 60 programs in 17 cities across the country
Number Served over 6,000 in spring 1999
Components Each community adapted one of four nationally recognized extended-service school models.

The Beacon
Originally implemented in New York City, Beacons Initiatives aim to: develop and operate school-based community centers; create “safe havens” for youth and families in poor neighborhoods; and promote youth development and resiliency. Youth development activities are offered in five core areas: education, recreation and enrichment, career development, leadership development, and health. Activities take place during nonschool hours and emphasize several factors important to youth resiliency: caring adult relationships, engaging activities, high expectations, youth's opportunity to make a contribution, and continuity. A lead agency at each Beacon Center manages activities. A local organization provides technical assistance in organizational development and youth development practices. An oversight committee, consisting of school district staff and executive staff from CBOs involved in the program, provides general policy and management oversight. Each school has a school-level decision-making body that includes parents and other community representatives.

Bridges to Success
Originally implemented in Indianapolis, Indiana, this program seeks to increase students' educational success by better meeting noneducational needs of children and families through a partnership of education, human service, and community service delivery systems, with a long-range vision of establishing schools as “lifelong learning centers” and community focal points. Each site provides activities that promote positive youth development during nonschool hours. Activities include educational enrichment, career development, arts and culture, life skills, counseling, case management, physical and mental health services, and recreation. The local United Way agency acts as the lead organization and fiscal agent. A local governance structure made up of United Way, school district, social service, and community representatives develops citywide programming strategies and oversees implementation. School-level councils assess community's needs and assets, and design and implement program interventions. Councils include a program coordinator, school principal and other school staff, parents, students, and local partners.

Community Schools
Originally implemented in the Washington Heights section of New York City by the Children's Aid Society, their mission is to provide educational excellence and human services through school, parent, and community partnerships, and to create seamless integration of school-day activities with extended-day programs. A wide range of youth development programming is offered during the school day and nonschool hours. Social services and parent education are also provided. Governance is through comanagement of school facilities by the school and a CBO. Local universities play a key role in technical assistance and planning. An oversight committee, consisting of university staff, executive staff from CBOs involved in the program, and school district staff, provide general policy and management oversight. Each school has a school-level decision-making body that includes parents and other community representation.

West Philadelphia Improvement Corporation (WEPIC)
Originally implemented in Philadelphia, Pennsylvania, this program aims to produce comprehensive, university-assisted community schools that serve, educate, and activate all members of the community, revitalizing the curriculum through a community-oriented, real-world, problem-solving approach. The program provides academically based community service activities, such as graduate and undergraduate interns who work in schools to provide educational assistance and mentoring to youth. School principals and staff play key decision-making roles, such as deciding what substantive areas will be addressed through the initiative. Community councils provide guidance on program content.
Funding Level $13 million (over five years)
Funding Sources the Wallace Foundation; school districts (and schools); federal, state, and local government agencies; and CBOs


Evaluation

Overview Public/Private Ventures and the Manpower Demonstration Research Corporation used a multi-method, multi-year evaluation approach designed to provide an understanding of the breadth of programming experiences and the ability to delve more deeply into particular issues related to the initiative's implementation and youth outcomes.
Evaluator Public/Private Ventures and Manpower Demonstration Research Corporation (subcontractor)
Evaluations Profiled Planning and Implementation Evaluations (Extended Service Schools: Putting Programming in Place and Challenges and Opportunities in After School Programs: Lessons for Policymakers and Funders)

Multiple Choices After School: Findings From the Extended-Service Schools Initiative
Evaluations Planned None—evaluation has ended.
Report Availability Grossman, J. B., Walker, K., & Raley, R. (2000). Extended Service Schools: Putting programming in place. Philadelphia: Public/Private Ventures.

Grossman, J. B., Walker, K., & Raley, R. (2001). Challenges and opportunities in after school programs: Lessons for policymakers and funders. Philadelphia: Public/Private Ventures.

Grossman, J. B., Price, M. L., Fellerath, V., Jucovy, L. Z., Kotloff, L. J., Raley, R., et al. (2002). Multiple choices after school: Findings from the Extended-Service Schools Initiative. Philadelphia: Public/Private Ventures.

Available at www.ppv.org/ppv/publications/publications.asp?search_id=19.


Contacts

Evaluation Jean Grossman
Senior Vice President
Public/Private Ventures
2000 Market Street, Suite 600
Philadelphia, PA 19103
Tel: 215-557-4474
Email: jgrossma@princeton.edu
Program Ed Pauly
Director of Evaluation
The Wallace Foundation
Two Park Avenue, 23rd Floor
New York, NY 10016
Tel: 212-251-9700
Email: epauly@wallacefunds.org
Profile Updated November 14, 2003

Evaluation 1: Planning and Early Implementation Evaluations (Extended Service Schools: Putting Programming in Place and Challenges and Opportunities in After School Programs: Lessons for Policymakers and Funders)


Evaluation Description

Evaluation Purpose To examine two areas related to planning and implementation: (1) first-year accomplishments, including activities put into place, participant characteristics and recruitment, program management, and implementation challenges; and (2) what it takes to launch these types of school/community initiatives, including who participated in the collaborative process, factors influencing the success of city-level collaborative efforts, school and program content selection processes, financing during early implementation, and the process of management devolution to the school level. Note that findings from this profile come from two reports: Extended Service Schools: Putting Programming in Place and Challenges and Opportunities in After School Programs: Lessons for Policymakers and Funders (see evaluation section above for detailed references).
Evaluation Design Non-Experimental: Data on summer and school year programs were collected from program staff, school staff, and program stakeholders in all 17 cities during the first year of operation. In-depth implementation data were collected from 10 research cities (at least two per model) during two rounds of site visits (two cities were only visited once).
Data Collection Methods Interviews/Focus Groups: During site visits to the 10 research cities, interviews were conducted with program staff, activity providers, initiative leaders, local funders, and school district personnel to gather information about program activities, management, and implementation.

Observation: Program activities were observed during the second site visits in five of the 10 research cities. Several key dimensions of program quality were examined in these observations: youth's relationships with adults and peers, structure and management of activities, and opportunities for youth decision making. A total of 23 activities were observed, most of which lasted between 30 and 60 minutes.

Secondary Source/Data Review: Enrollment and participation data were collected in five of the research cities, chosen to be intensive research cities.

Surveys/Questionnaires: To learn about the range of programming experiences in all cities, program directors and school coordinators in all 17 cities were asked to fill out annual organizational surveys, which gathered information about program characteristics, including programming, staff, cost, and financing data (although few cities provided information on costs). Approximately three-quarters (n = 45) of school coordinators responded to the survey. In addition, in the five intensive research cities, parents filled out a brief questionnaire when they enrolled their child. Parents provided demographic data, as well as their reasons for enrolling their children. Data from approximately 800 parents were available at the time of the analysis.
Data Collection Timeframe Data were collected from late 1998 to late 1999.


Findings:
Formative/Process Findings

Activity Implementation According to the evaluators, the planning period was an important step in getting programs ready to open their doors. It was a period for relationship building between partners and completion of three important tasks: choosing schools, assessing needs, and finding resources (financial and other).

Even though the four models emphasize different goals, the programming on the ground was remarkably similar their first year. They all provided a strong and consistent commitment to academic work. Evaluators speculate that this occurs because of the relative ease with which these programs, particularly homework help, can be set up, and because most partners (including parents) agree that academic support is of primary importance. During the first school year, approximately 40% of activity hours were academic, 20% were cultural or creative enrichment activities, 20% were athletic, and the remainders were various. Summer program offerings were evenly split among these four areas, reflecting the reduced urgency of academic pursuits during the summer and youth's desire to be more physically active.

Positive adult/youth relationships was consistently the strongest of the activity quality areas observed. Staff worked hard to make time with the youth both fun and meaningful. Low student-to-staff ratios promoted highly valued one-to-one interaction.

Effective structure and management of the youth participants involved setting up age-appropriate behavioral demands that were clear to the participants and struck a balance between being firm yet warm. With middle school and high school students this often meant being flexible and somewhat hands-off. However, highly structured activities for older youth “worked” if the youth adopted them as their own. For younger children, good management required more structure. Staff were seen as needing to keep control and manage disputes, give them ideas, provide resources, and help them when they “got stuck.”

Youth decision-making opportunities were not consistently observed in activities because, according to the evaluators, it requires planning and often staff training. When such opportunities were seen, they involved youth helping plan an activity or helping decide how to carry out an activity.

Youth leadership was infrequently observed and typically involved situations where youth acted as group leaders or captains, tutored each other, or informally made group decisions. Middle and high school students had more formal opportunities for leadership than elementary students.
Costs/Revenues Assembling resources to start and sustain programs was a major challenge for collaborative members. In general, programs assembled the needed resources through a combination of commitments from community organizations; in-kind donations by citizens and businesses; the use of the school building for program activities; grants from foundations; and donated or subsidized time from community leaders and school personnel.

Generally, programs used their monetary resources to fund core staff's time, to subsidize or fully pay for some youth activities, and to cover some administrative costs. Costs covered with redirected funds or in-kind donations exceeded the expenses paid for by cash contributions by 100–200%.

A review of the expenditures of a sample of 11 ESS cities showed that a school's program could be started on less than $30,000 cash annually, provided it could leverage other resources and the use of a school building.

In almost all ESS sites, sufficient funding was not allotted for after hours busing services, which was found to entail unexpectedly high costs.
Program Context/Infrastructure Three types of school-level governance were tried: lead-agency led, small-team governed, and shared. While the shared structure was effective in generating community interest in the initiative, it proved difficult to sustain as the initiative progressed. Almost all the schools with this governance structure shifted to more heavy reliance on either a lead-agency-led structure or a small-team structure. Lead-agency and small-team governance proved to be effective for decision making at the school level.

City-level oversight committees provided important human resources to the initiative. These committees usually include very senior staff and high-level political personnel that could often find solutions to specific problems that the school level governance group could not handle.

The results of the needs assessments and community mapping undertaken by the cities were crucial to building support for the initiative, especially among school principals, and to help the program design, but only if the needs of each school was assessed separately.

Program planners and stakeholders benefited from the opportunity to visit active programs in other cities and this helped to transmit a concrete vision of how after school programs operate.

The transition from planning to implementation was challenging in many sites. Shifts in decision making and management authority from the planning team to school-level government structures and CBOs were particularly difficult. In general, two major factors contributed to the challenges of transferring authority: first, some school-level governance structures were unprepared to take on key decision-making tasks surrounding the ESS program at their school; and second, local governance teams and agency personnel sometimes lacked clarity about their respective roles and responsibilities.
Program-School Linkages Schools were mostly not the underused resources policymakers thought they were. Programs often had to compete for school space, especially gyms and computer labs, with activities being run by teachers, students, sports teams, and outside organizations. Tight school budgets and scarce custodial resources meant that school administrations' tended to limit access to even the free space, when the upkeep of the space or the cost of potential damage was high (such as with computer labs). Access was improved as the program staff built trusting relationships with key school staff.

Small-team governance, where principals are included on the team, strengthened school participation in the local initiative, which may serve to create and sustain strongly integrated school/community partnerships.

A common criterion for selecting schools was the principal's support. However, a large number of schools had new principals by the time implementation began.
Recruitment/Participation Across sites, enrollees were 51% female (49% male) and their ethnic and racial makeup was: 45% African American, 33% white, 19% Latino, and 3% other.

In the intensive research cities, 26% of enrollees lived in single parent households, 66% received free or reduced-price lunches, and 46% lived in households with incomes of $20,000 or less.

Transportation challenges impeded recruitment. Given the limited supply of buses and drivers, as well as their cost, busing after hours was scarce. Youth who could not walk home and lacked adults to pick them up often could not participate. ESS coordinators and partners consistently expressed concern about transportation challenges, especially because students who required after-hours busing were often those who could most benefit from programs. The immediacy of this issue pressed programs to try to quickly develop solutions, one of which was to obtain after-hours busing.

Recruitment strategies undertaken by program staff included: making presentations in classrooms and assemblies; setting up information tables at Back-to-School and Open House nights; interacting with youth in school lunch rooms and halls; holding special events for families; inviting parents for coffee; distributing and posting fliers and brochures in multiple languages; making home visits; and enlisting partners and collaborators to advertise the program through their own networks. In addition, staff consistently reported a strong reliance on informal communication, especially with students' teachers.

Since so much of the recruitment was school-based, the quality of relationships between school and program staff had a direct impact on the ability of programs to recruit students.

Preliminary data suggest that while ESS programs reach thousands of children who live under disadvantaged conditions, additional effort was needed to attract disadvantaged students. According to a number of program coordinators, many early enrollees were students who seemed to be more assertive and involved in school and other activities. They also noted that parents who were most involved with their children were the ones who responded first to the enrollment opportunity.

Referrals from school staff and student support teams were the most common means of recruiting more disadvantaged youth. Several sites developed targeted recruitment strategies designed to be less stigmatizing than referrals, such as holding registration sessions in low-income housing units where students lived, or associating the program with their lead agencies rather than the school to attract youth who might be wary of school. Some programs also switched from a first-come, first-serve basis to a multiple-week sign-up period to enable more children to apply. Overfilled activities were filled by randomly picking from the applicants.

Two factors were central to the success of recruitment: the age of youth (younger youth were easier to recruit) and the number of other youth-serving organizations and programs in the community (the more of these programs, the more difficult recruitment was).

Middle and high school students were found to be harder to recruit due to their busier schedules, increased responsibilities, and greater freedom. Despite these challenges, coordinators devised several programming ideas to attract teens and have them participate consistently. Ideas that seemed to work best included flexible open-door policies and opportunities for leadership and loosely guided autonomy. Hiring staff who were closer in age and share similar ethnic or racial backgrounds with the middle and high school students also great helped recruitment.

Parents supplied the following reasons for enrolling their child in ESS: their child wanted to get involved (76%); their child can make friends and have fun (54%); it will help their child do better in school (53%); it is a safe place for their child after school (41%); their child can get to and from the program easily and safely (30%); it provides dependable after school care (21%); it provides affordable after school care (19%); and school staff suggested that their child enroll (12%).
Staffing/Training Although many schools tried to use a half-time school coordinator, programs learned that this was a full-time job. Project directors (who oversaw all the school coordinators) expressed concern about both the program's quality and coordinator burnout when they were only half time. To meet implementation challenges, some programs switched school coordinators from part time to full time. When programs operated more than eight hours a day, an assistant director was needed to cover the extra time.

The experience and education of program coordinators did not seem as important in their ability to operate the programs as their relationships with school personnel.

Youth in focus groups reported that they liked staff presence because adults could mediate potential disputes among youth and provide ideas and resources. They also reported that they wanted adult support, but didn't want staff to be constantly “on” them.
Systemic Infrastructure Partners were initially attracted to ESS for a variety of reasons: about a third saw ESS as an opportunity to strengthen school-community partnerships; another third saw it as an opportunity to expand their own programming; and the final third came simply because they were invited.

Cities with a history of collaboration moved more quickly in implementing their programs and had fewer tensions than did the one city with limited prior history.

Cities noted two specific areas in which technical assistance provided by national intermediaries proved extremely useful in creating their collaborations: clarifying decision making and promoting the development of positive relationships among partners.

The opportunities intermediaries gave cities in the planning stage to visit other cities with active initiatives and to go to cross-site conferences proved to be a particularly useful way to illustrate the potential of ESS and generate enthusiasm for the local initiatives.

The results of needs assessments and community mapping undertaken by the cities were crucial to building support for the initiative, especially among school principals. The results' usefulness in guiding program planning was less clear, especially in communities that involved multiple schools.

School selection was guided by a variety of criteria ranging from the academic performance and backgrounds of the school's population, to geographic considerations, to principal support. Principal support was almost always a key consideration.

When cities had multiple schools involved in their program, establishing oversight committees permitted them to more easily coordinate policies and activities across schools.
Lessons for Policymakers and Funders
The evaluators provide several recommendations for policymakers and funders, based on the findings of this evaluation. According to the evaluators, expectations for after school programs should be tempered by well-established knowledge about what youth programs can and cannot achieve and under what timeframe and what circumstances. Specifically, the lessons learned from this evaluation are as follows:

  • After school programs do give students access to caring adults, opportunities to interact socially with peers, and chances to learn new skills.
  • While academic activities are a substantial part of after school programs, many children attend only a day or two a week. Changing grades and test scores will take will thus take time.
  • Providing supervised, safe after school environments may lower the rate of youth victimization and lower neighborhood delinquency, but more serious juvenile crime is committed by older youth, who at least currently are not likely to be enrolled.

Evaluation 2: Multiple Choices After School: Findings From the Extended-Service Schools Initiative


Evaluation Description

Evaluation Purpose To address the following four questions: (1) Which youth came to ESS and why? (2) What were the characteristics of high quality program activities (i.e., those that promoted positive youth development)? (3) What benefits did youth gain from participation? and (4) What was the cost to operate programs, and how were they financed?
Evaluation Design Quasi-Experimental and Non-Experimental: Researchers conducted interviews at multiple in-depth site visits in six cities (the intensive research cities). Quantitative data (attendance records, cost data, intake form data, and surveys) were collected in 10 schools in those six cities (the intensive study schools). Student surveys were administered to fourth through eighth graders as the student enrolled in the program and in the spring of 2001 to examine program outcomes based on levels of participation. All fourth through eighth grade students who ever enrolled in the program were invited to take the follow-up survey. They did not have to be currently enrolled in the program. In three of the middle schools, they conducted observations of the after school activities. The 10 intensive study programs enrolled 1,511 first through eighth graders in the study from the beginning of fall 1999 through the end of fall 2000. Study enrollment required parental consent.
Data Collection Methods Interviews/Focus Groups: Staff, partners, students, parents, and key city officials were interviewed at the six in-depth cities during the 2000–2001 school year to gain insight into individual sites and learn about promising practices. In three of the 10 intensive study programs, a total of 30 child-teacher-parent triads interviews were conducted to collect in-depth data about the quality of activities and the ways the program fit into children's lives.

Observation: In three of the intensively studied middle schools, evaluators observed 30 ESS activities during the 2000–2001 school year. They used an observation instrument to rate the quality of a set of constructs or dimensions (e.g., youth's relationships with adults and each other, decision-making and leadership opportunities, engagement level) and described the behaviors and practices they observed for each dimension. Characteristics were rated on a five-point scale, with five being the highest and ratings of two or less connoting the presence of something negative. Each activity was observed in its entirety two or three times, and ratings were averaged to come up with an activity's mean score for each characteristic.

Secondary Source/Data Review: During the 2000–2001 school year, attendance records and cost data were collected from the 10 intensive study schools. In addition, parents completed intake forms at the time of enrollment that provided demographic information and reasons for registering their children in ESS. A total of 2,047 intake forms were collected for youth in first through eighth grade (although not all ended up attending ESS).

Surveys/Questionnaires: Evaluators administered a baseline survey to fourth through eighth graders between fall 1999 and December 2000 as they enrolled in the program. The baseline survey, which was completed by 786 youth, asked about youth's beliefs, attitudes, and behaviors. In spring 2001, a follow-up survey was administered to fourth through eighth graders who had ever enrolled in ESS (although not necessarily currently participating) and were still in the school. Because many original enrollees left the school, only 674 completed the follow-up survey, of which 371 had also completed a baseline survey. This represents at least a 63% response rate (without adjusting for school mobility rates). The follow-up survey included questions on 16 outcome measures of behaviors and social-psychological constructs across three outcome areas: risk and non-risk behaviors, school attitudes and behaviors, and adult support; as well as many questions about what and why they attended various types of after school activities. Thus, all analysis held constant non-ESS extracurricular activity attendance. They tested the hypothesis that youth who attended more hours of structured and supervised activities (through ESS and other programs) would show more developmental gains than those who participated in fewer such activities.

A telephone survey of participants' parents was conducted to learn about their views of ESS. The survey was administered to 221 parents from a pool of 336 who primarily spoke English or Spanish and whose child was in at least fourth grade or nine years old at the time of the survey and attended ESS at least one day during the 2000–2001 school year.

To learn about program activities, school site coordinators and city-level program directors completed annual organizational surveys summarizing what their programs were doing.
Data Collection Timeframe Data were collected during the 1999–2000 and 2000–2001 school years.


Findings:
Formative/Process Findings

Activity Implementation Among the 30 activities observed, all but two provided at least some developmental supports and opportunities for youth, although types of support and opportunities varied.

Enrichment activities provided the highest-rated environment for positive youth development; average ratings for adult-youth relationships, cooperative peer interaction, and decision making were all higher for this type of activity than for others.

Creative arts activities scored highly on youth engagement overall and were individually strong in their provision of developmental opportunities.

Remedial academic activities (homework help and tutoring) offered fewer kinds of developmental supports and opportunities than other activities; they provided fewer opportunities for peer interaction and decision making and rated low in youth engagement. Still, they achieved a purpose; staff gave youth support to complete tasks successfully, and youth reported that sessions helped them catch up and complete homework.

In more than half of activities observed, the quality of peer interactions was assessed to be very positive. Youth seemed comfortable together, laughed and joked, and seemed to enjoy each other's company. In addition, the quality of adults' relationships with the youth seemed to affect the quality of peer interactions in an activity. For example, teachers who made the effort to instill a spirit of teamwork set a positive example by being polite and respectful to the youth and, similarly, discouraged them from making negative comments about each other. Youth picked up on this and treated each other with respect.

Only a third of observed activities provided opportunities for high levels of youth collaboration and teamwork. Small group projects provided the most opportunities while academic and visual art activities where youth worked individually provided the least.

Only a few of the observed activities provided opportunities for youth decision making: six were rated four or better in this area (on a scale of one through five, with five being the best), while 15 received ratings less than two. Youth-driven and open-ended activities (the final product or goal was not predetermined by an adult) offered youth the most decision-making opportunities. These were generally cooperative group activities where youth worked together on projects.

Only seven activities were observed to provide leadership opportunities for youth. These opportunities tended to be informal (usually those that encouraged youth to collaborate) and involved youth teaching or instructing peers. A negative relationship was found between the amount of adult instrumental support and youth leadership opportunities.

Of the 30 activities observed, 27 received positive or very positive youth engagement ratings. During activities, youth actively participated, seemed interested in completing the work, and were focused and attentive most of the time. Youth were more often distracted or complained of being bored in remedial academic and community service activities.

Among the different types of activities, sites had the most difficulty designing and implementing service projects that were challenging and engaging. Of the four service activities observed, three were relatively unchallenging and did not hold youth's interest.

Youth engagement was not correlated with youth decision making, suggesting that interpersonal and skill-building dimensions of an activity may be more important to youth than their degree of input into the activity.
Costs/Revenues Programs received a grant of $25,000–$50,000 for a six- to nine-month planning period. Once implemented, programs operated the first year on an additional $80,000–$300,000.

The 10 intensive study programs cost about $150,000 per school year (excluding use of space) on average, ranging from about $63,000–$266,000, to serve 63 youth after school five days a week. This translates into an average cost of about $15 per day per youth slot, with a range across programs of $8–$36. This range resulted from a variety of program factors, including community setting, types of activities, staff ratios, administrative structures, and investment in such factors as fundraising and future sustainability.

Almost half of programs' funding (45%) came from foundation grants, of which more than 86% was from the Wallace Foundation. Foundation funding was usually given as cash grants and was primarily used to pay salaries. School districts (and schools) contributed 21% of funds, government agencies gave 17%, and CBOs gave 10%.

Cash budgets covered 60% of total costs across the programs. Partner organizations covered the rest (ranging from 5%–58%) with redirected (non-cash) contributions. Cash expenditures went primarily to administrative costs, the largest portion of which went to staff salaries ($48,000 on average per site). Programs needed another $8,000 to cover such cash expenditures as office expenses, staff training, and public relations.

Redirected contributions primarily covered the costs of additional administrative expertise, youth leader salaries, custodial help, transportation, and snacks.

Across sites, the largest cost component was the administration of programs—primarily the salaries of the director and school coordinator (54% of total cost). The cost of providing youth activities (hiring other activities providers and providing equipment and materials) was among the most expensive items (35% of total cost). The cost of support services (transportation, snacks, and custodial help) accounted for only about 11% of costs.

Most sites obtained support services by developing strong partnerships with host schools and districts. In the 10 intensive study programs, these partners contributed, on average, over 20% of program costs (in addition to the rent-free use of school space), including some or all of the cost of transportation, custodial assistance, and snacks. On average, ESS programs spent only about $5,000 in cash expenditures on all three items.

Across sites, sustaining programs after the initial grant funding seemed likely to be a challenge, although several strategies were promising: using the Bridges to Success model, which comes with local United Way funding; having strong lead agencies for whom ESS fits a need; developing strong partnerships with other providers and funders; and collaborating with other youth initiatives to work toward the long-term goal of dedicated state funding.

What seemed to matter most in administrative costs was that sufficient resources were expended on internal management, developing and maintaining partnerships, and creating a solid funding base. Ideally, the site coordinator was full time and the program director was budgeted for enough time to develop partnerships, raise funds, and perform other necessary tasks. When either role was not sufficiently filled, programs suffered. Problems included a drop in youth attendance and a lack of system building for sustainability.

A key factor in the cost of activity leaders was the extent to which the school's teachers were involved in ESS and how salaries were covered. Some programs hired non-teachers at a lower pay scale; some sought the expertise of certified teachers even if it meant higher salaries; some covered teachers' higher salaries through redirected school district funds; and some paid teachers lower hourly salaries than they received during the school day.

Sites that served a large number of youth for fewer days per week incurred higher administrative costs because they had to handle transportation and other logistics of having a different group of youth scheduled for each day.

Some programs offered activities, particularly performance and visual arts activities, that were more costly to run, but enriched the program. Typically, they were able to do this because of their success in forming partnerships and leveraging resources.

Some programs had higher costs due to decisions to provide mentoring or one-on-one tutoring, or simply to have more staff available for youth. Some programs found ways to keep costs down, by hiring high school or college students who received minimum wage. In one program, graduate students volunteered as mentors and tutors for individual youth.

Program costs were influenced by community context, such as whether youth could walk home. Some sites incurred costs for transporting most youth home every day, especially in some of the larger school systems, where students were bused to school from many different neighborhoods. Transportation was less of an issue in smaller school districts and elementary schools, where many students lived within walking distance of school.

There was a range of costs for required services such as custodial help and snacks, depending on how services were covered. Costs of snacks depended partly on whether programs obtained them through the schools' free and reduced-price lunch program.

Sites that planned for the number of youth expected to attend activities rather than the number registered avoided extra resource costs. Some sites solved the problem by over-enrolling, knowing not all would attend, or by keeping waiting lists to replace dropouts.
Recruitment/Participation The 10 sites served an average of 63 youth a day. In most sites, different youth attended on different days of the week, so the number of individual youth served is much higher.

Across the eight sites that collected participation data on all youth enrolled in ESS, slightly more than half of the schools' total populations were attending ESS programs.

In terms of participants' family characteristics, nearly three-quarters of ESS participants (73.9%) had family incomes of $30,000 or less; almost the same proportion (71.8%) were eligible for free or reduced-price lunch; and 41.1% lived in a single-parent household. In addition, 22.4% of participants' parents did not graduate from high school, 30.0% had some college, and 23.7% were college graduates.

In terms of participants' demographic characteristics, their ethnic breakdown was: 40.1% white, 32.7% African American, 19.4% Hispanic, 7.7% Asian/Pacific Islander; 3.0% Native American; and 3.8% other (categories are not mutually exclusive). They were 54.7% female (45.3% male), and their average age was 10.3 years old.

ESS staff conducted outreach and registration activities in a variety of ways, with the goal to maximize students' and parents' knowledge of programs and increase the convenience of registering. Strategies included: sending notices about ESS to parents; calling parents or talking to them when they picked up their children; holding open houses in the evenings; holding information and registration sessions in public housing complexes where many students lived; and translating notices for non-English-speaking parents.

By their second year, eight of the 10 intensive study schools reported operating at capacity (serving as many youth as possible with available resources); three programs capped enrollment and one limited the number of days youth could register to deal with this issue.

As programs developed, staff targeted recruitment strategies toward attracting high-needs students (those who were failing courses, were disengaged from school, and had behavior problems). Strategies included: working with school staff to identify and encourage such students to enroll; having teachers recommend ESS to parents; reserving enrollment slots for high-needs youth referrals (especially for capped programs); hiring staff who tended to be similar to youth in gender, race, and age; and offering opportunities for less structured activities to attract youth who are more comfortable in a casual environment. These efforts seemed to result in gains, as suggested by an increase during the two years of the study in the proportion of ESS youth eligible for free or reduced-price lunch (from 66% to 72%).

Students tended to participate in ESS over an extended period of time, not just a single semester. More than a third (35%) of the enrollees participated all four semesters that were covered by this study and, overall, 84% participated in two or more semesters.

On average, participants were registered for 33 program days per semester, or roughly three days a week, ranging from 18.6 to 84.6 across the 10 sites. They attended almost two-thirds (63%) of those days (20 days per semester, or about two days a week), with a range of 34.4% to 73.9% (10–62 days per semester). Three factors contributed to variations: youth's ages, participation requirements, and youth and parent preferences for how youth spend after school time. Averages were strongly affected by high participation levels at one school. Excluding that school, the average number of scheduled days drops to 27 and average days present drops to 15.

ESS tended to attract youth who were already “doers.” During the year before enrolling in ESS, 70% of youth went to organized after school activities: 18% went once a week, 24% went two or three times a week, and 28% went four or five times a week. However, 30% of the participants had participated in no after school activities in the previous year. Youth who had previously been most involved in organized activities after school (participating at least twice a week) primarily substituted ESS for their other activities. ESS participants who had previously participated in no organized activities were enrolled in ESS an average of 2.2 days a week, and youth with one day of previous activities were enrolled an average of 1.8 days a week. Both groups reported attending more than a day a week of other after school activities.

Given that students chose to come to this free after school program only about a day a week, a key question was why. Both students and parents were asked. Reasons youth gave for not attending ESS more often were: 28% had other things to do; 18% were not interested in activities or their friends did not attend; 15% did not have a way to get home; and 11% did not like the treatment by staff or other students. Youth who attended the fewest days were more likely to give the nonattendance reason that they had other things to do; youth also made it clear that there were a number of ways they spent after school time that could draw them away from ESS. Older youth were more inclined to give the reason for nonattendance that they were not interested in the activities. Also, older youth indicated more often that they went home or to a friend's house after school.

Reasons parents gave for their children not attending ESS more often were: their child had other things to do (42%); program policies limited participation (42%); their child did not like the activities or their friends did not attend (23%); their child did not like the treatment by staff or other students (15%); and a lack of transportation (15%). Two of the reasons, child's dislike of activities and lack of transportation, were cited much more often by parents of children with low attendance.

The follow-up survey confirmed that the youth were busy with other activities. Youth reported spending after school hours during the previous week (youth may have spent time in different ways in a single day): at their own home or home of a friend, sitter, or relative (3.2 days, on average); in organized activities at school, such as ESS (1.8 days); in organized activities at nonschool locations (1.4 days); and in school for other reasons, such as using the library or talking to a teacher (0.5 days).

Program policy, however, did affect attendance rates. Programs that required registration for a greater number of days per week were able to serve youth more intensively, but served fewer overall. In addition, required four- or five-day-a-week enrollment increased the number of days scheduled and attended, but resulted in low attendance rates unless there was a well-articulated and enforced attendance policy.

Programs that allowed selective registration for activities offered only on certain days involved more youth and may have attracted youth with more diverse interests and of more diverse ages. However, since individual youth participated fewer days, they were less exposed to the academic and developmental supports and opportunities. In addition, flexible enrollment policies limited the usefulness to parents with childcare needs.

Younger enrollees were scheduled for more program days and had higher attendance rates than older enrollees. First through third graders attended 28 days on average (73% of their scheduled days), while sixth through eighth graders attended only 13 days (a 54% attendance rate). In addition, a much higher percentage of younger enrollees attended more days during a semester; 46% of first through third graders attended 20 or more days, while only 23% of sixth to eighth graders attended 20 or more days, and almost a quarter attended four days or fewer.

Parents' concerns seemed to influence their children's attendance patterns, and those concerns tended to correlate with their children's ages. When asked why they registered their child in ESS, parents of first through third graders were far more likely than parents of middle school students to say that they had done so because the program offered opportunities for academic improvement and provided a safe place for their child to be after school. It was parents who cited these reasons for enrollment whose children tended to attend more days of ESS than other participants.


Summative/Outcome Findings

Academic At follow-up, approximately two-thirds (65%) of participants surveyed believed ESS helped them do better in school, and even more parents (79%) found the program helpful in this way.

Half (50%) of the students reported liking school better due to the program, whereas 86% of parents felt their children liked school more as result of ESS.

Of those surveyed, 71% of participants reported that ESS helped them learn that hard work pays off, and 82% of parents reported that ESS helped their child try harder in school.

Seventy-one percent of parents surveyed reported that ESS helped their child complete homework.

Statistical analysis of the change in outcomes of youth participating at different levels finds that over approximately 12 months, participation in ESS was associated with positive changes in school belonging and a positive change in school behaviors—namely paying attention in class and not skipping school. Comparing more frequent to less frequent ESS participants, one additional day of ESS attendance per week was associated with a 27% increase in the likelihood that they would have “a very high sense” of school belonging (p<.10), a 52% increase in likelihood that a child would report almost always paying attention in class (p<.01), and a 31% decrease in likelihood that a child would start skipping school (p<.05). This translates into: only 11% of the average two-day-a-week ESS attenders starting to skip school versus 20% of the non-attenders; 49% of the average two-day-a-week ESS attenders really paying attention in class versus 29% of the non-attenders; and 84% of the average two-day-a-week ESS attenders feeling likely to felt very proud of being a part of the school versus 76% of the non-attenders. However, statistical analysis revealed no significant relationship between program attendance and the likelihood that a youth had not finished homework nor the change in the student's sense of academic mastery.
Family Of parents surveyed, 47% said ESS let them attend classes or job training more easily; 45% said ESS helped them get a better job or do better at their job; 57% said their child's participation in the program helped them manage their own work schedule; and 86% said ESS helped them to better appreciate their child's talents.

There were no significant relationships between program attendance in ESS and parents' involvement with their children.

Of parents surveyed, 74% said ESS helped their child get along with family members.

The majority of ESS parents surveyed (80%) said they were less worried about their child's safety after school.
Prevention At follow-up, 48% of participants surveyed reported that ESS helped them deal calmly with people who are mean or start fights; 72% of parents agreed that ESS helped their child settle arguments without fighting. When asked if ESS helped the youth stay out of trouble, 65% of participants surveyed and 84% of parents believed ESS was very useful in this regard. Sixty-two percent of participants surveyed agreed that the program helped them learn how to say no to things that they know are wrong or that make them uncomfortable.

Statistical analysis of the change in outcomes of youth participating at different levels finds that over approximately 12 months, participation in ESS was associated with a decrease in the likelihood that the student started drinking alcohol and an increase in their ability to deal with their anger in socially appropriate manner (not hitting or fight, but talking). Comparing more frequent to less frequent ESS participants, one additional day of ESS attendance per week was associated with a 28% decrease in likelihood that a child would start to drink (p<.10) and 54% increase in the likelihood that the student would deal with their anger in socially appropriate manner (p<.01). This translates into: 9% of the average two-day-a-week ESS attenders starting to drink versus 16% the non-attenders; and 73% of the average two-day-a-week ESS attenders dealing with anger appropriately versus 53% of the non-attenders.

No significant relationships were found between ESS participation and the likelihood that over the past month youth had hit someone, vandalized school property, been sent to the principal, or stolen things in school.

Comparing more frequent to less frequent ESS participants, one additional day of ESS attendance per week was associated with a 30% decrease in likelihood that a child would have been bullied over the past month (p<.11).
Youth Development At follow-up, 61% of participants surveyed said that being in ESS helped them feel less shy around adults, and 63% reported that ESS helped them be less shy around other kids.

Of participants surveyed, 73% felt that ESS helped them make friends; 92% of parents agreed that the program helped their child make friends.

The majority of parents surveyed (86%) felt that that the program helped their child get along better with other children.

Of participants surveyed, 52% felt that ESS helped them learn about other cultures.

Seventy-four percent of participants surveyed agreed with the statement: “I see that I have choices and possibilities in life that I didn't know I had,” as a result of ESS.

Seventy-two percent of participants surveyed reported that the program helped them do things they didn't think they could do.

The majority of parents (91%) surveyed felt that the program helped their child learn new skills or develop new interests.

Of participants surveyed, 69% felt that ESS helped them feel good about themselves.

Of parents surveyed, 89% felt that ESS helped their child feel more self-confident.

Comparing more frequent to less frequent ESS participants, one additional day of ESS attendance per week was associated with a 58% increase in likelihood that a child would have done volunteer work (p<.01) and a 27% increase in likelihood that a child would have written a poem, story, or play or written in a journal or diary, not for school (p<.10) over the past month.

No significant relationships were found between program participation and the likelihood that over the past month a youth had performed in front of others (such as reading a poem, singing, dancing, or giving a speech) or read a book from beginning to end that wasn't assigned for school.

No significant relationships were found between program participation and the likelihood that youth would have more adult confidantes or adults who provided emotional support.

© 2016 Presidents and Fellows of Harvard College
Published by Harvard Family Research Project