The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.
HFRP
> Out-of-School Time
> OST Database & Bibliography
> Database
> Extended-Service Schools Initiative
> Evaluation #1 (1998-1999): Planning and Early Implementation Evaluation
A Profile of the Evaluation of Extended-Service Schools Initiative
Profile Information
- Full Text (HTML)
- How to cite
- About the OST Database
Program Description
| Overview | The Extended-Service Schools Initiative (ESS) supports organizations in 17 cities to provide high quality youth development activities in school buildings during nonschool hours. Services are provided through partnerships between a local low-income public school district and a community-based organization (CBO) and/or a university. The 17 cities are: Oakland and Long Beach, California; Savannah and Atlanta, Georgia; Denver and Aurora, Colorado; Guilford County and Highpoint, North Carolina; Albuquerque, New Mexico; Salt Lake City, Utah; Central Falls, Rhode Island; Minneapolis, Minnesota; Mesa, Arizona; Boston, Massachusetts; Missoula, Montana; Flint, Michigan; and Jacksonville, Florida. |
| Start Date | 1998 |
| Scope | national |
| Type | after school, summer/vacation, before school, weekend, comprehensive services |
| Location | urban |
| Setting | public schools |
| Participants | kindergarten though high school students, other community members |
| Number of Sites/Grantees | 60 programs in 17 cities across the country |
| Number Served | over 6,000 in spring 1999 |
| Components | Each community adapted one of four nationally recognized extended-service school models. The Beacon Originally implemented in New York City, Beacons Initiatives aim to: develop and operate school-based community centers; create “safe havens” for youth and families in poor neighborhoods; and promote youth development and resiliency. Youth development activities are offered in five core areas: education, recreation and enrichment, career development, leadership development, and health. Activities take place during nonschool hours and emphasize several factors important to youth resiliency: caring adult relationships, engaging activities, high expectations, youth's opportunity to make a contribution, and continuity. A lead agency at each Beacon Center manages activities. A local organization provides technical assistance in organizational development and youth development practices. An oversight committee, consisting of school district staff and executive staff from CBOs involved in the program, provides general policy and management oversight. Each school has a school-level decision-making body that includes parents and other community representatives. Bridges to Success Originally implemented in Indianapolis, Indiana, this program seeks to increase students' educational success by better meeting noneducational needs of children and families through a partnership of education, human service, and community service delivery systems, with a long-range vision of establishing schools as “lifelong learning centers” and community focal points. Each site provides activities that promote positive youth development during nonschool hours. Activities include educational enrichment, career development, arts and culture, life skills, counseling, case management, physical and mental health services, and recreation. The local United Way agency acts as the lead organization and fiscal agent. A local governance structure made up of United Way, school district, social service, and community representatives develops citywide programming strategies and oversees implementation. School-level councils assess community's needs and assets, and design and implement program interventions. Councils include a program coordinator, school principal and other school staff, parents, students, and local partners. Community Schools Originally implemented in the Washington Heights section of New York City by the Children's Aid Society, their mission is to provide educational excellence and human services through school, parent, and community partnerships, and to create seamless integration of school-day activities with extended-day programs. A wide range of youth development programming is offered during the school day and nonschool hours. Social services and parent education are also provided. Governance is through comanagement of school facilities by the school and a CBO. Local universities play a key role in technical assistance and planning. An oversight committee, consisting of university staff, executive staff from CBOs involved in the program, and school district staff, provide general policy and management oversight. Each school has a school-level decision-making body that includes parents and other community representation. West Philadelphia Improvement Corporation (WEPIC) Originally implemented in Philadelphia, Pennsylvania, this program aims to produce comprehensive, university-assisted community schools that serve, educate, and activate all members of the community, revitalizing the curriculum through a community-oriented, real-world, problem-solving approach. The program provides academically based community service activities, such as graduate and undergraduate interns who work in schools to provide educational assistance and mentoring to youth. School principals and staff play key decision-making roles, such as deciding what substantive areas will be addressed through the initiative. Community councils provide guidance on program content. |
| Funding Level | $13 million (over five years) |
| Funding Sources | the Wallace Foundation; school districts (and schools); federal, state, and local government agencies; and CBOs |
Evaluation
| Overview | Public/Private Ventures and the Manpower Demonstration Research Corporation used a multi-method, multi-year evaluation approach designed to provide an understanding of the breadth of programming experiences and the ability to delve more deeply into particular issues related to the initiative's implementation and youth outcomes. |
| Evaluator | Public/Private Ventures and Manpower Demonstration Research Corporation (subcontractor) |
| Evaluations Profiled | Planning and Implementation Evaluations (Extended Service Schools: Putting Programming in Place and Challenges and Opportunities in After School Programs: Lessons for Policymakers and Funders) Multiple Choices After School: Findings From the Extended-Service Schools Initiative |
| Evaluations Planned | None—evaluation has ended. |
| Report Availability | Grossman, J. B., Walker, K., & Raley, R. (2000). Extended Service Schools: Putting programming in place. Philadelphia: Public/Private Ventures. Grossman, J. B., Walker, K., & Raley, R. (2001). Challenges and opportunities in after school programs: Lessons for policymakers and funders. Philadelphia: Public/Private Ventures. Grossman, J. B., Price, M. L., Fellerath, V., Jucovy, L. Z., Kotloff, L. J., Raley, R., et al. (2002). Multiple choices after school: Findings from the Extended-Service Schools Initiative. Philadelphia: Public/Private Ventures. Available at www.ppv.org/ppv/publications/publications.asp?search_id=19. |
Contacts
| Evaluation | Jean Grossman Senior Vice President Public/Private Ventures 2000 Market Street, Suite 600 Philadelphia, PA 19103 Tel: 215-557-4474 Email: jgrossma@princeton.edu |
| Program | Ed Pauly Director of Evaluation The Wallace Foundation Two Park Avenue, 23rd Floor New York, NY 10016 Tel: 212-251-9700 Email: epauly@wallacefunds.org |
| Profile Updated | November 14, 2003 |
Evaluation 1: Planning and Early Implementation Evaluations (Extended Service Schools: Putting Programming in Place and Challenges and Opportunities in After School Programs: Lessons for Policymakers and Funders)
Evaluation Description
| Evaluation Purpose | To examine two areas related to planning and implementation: (1) first-year accomplishments, including activities put into place, participant characteristics and recruitment, program management, and implementation challenges; and (2) what it takes to launch these types of school/community initiatives, including who participated in the collaborative process, factors influencing the success of city-level collaborative efforts, school and program content selection processes, financing during early implementation, and the process of management devolution to the school level. Note that findings from this profile come from two reports: Extended Service Schools: Putting Programming in Place and Challenges and Opportunities in After School Programs: Lessons for Policymakers and Funders (see evaluation section above for detailed references). |
| Evaluation Design | Non-Experimental: Data on summer and school year programs were collected from program staff, school staff, and program stakeholders in all 17 cities during the first year of operation. In-depth implementation data were collected from 10 research cities (at least two per model) during two rounds of site visits (two cities were only visited once). |
| Data Collection Methods | Interviews/Focus Groups: During site visits to the 10 research cities, interviews were conducted with program staff, activity providers, initiative leaders, local funders, and school district personnel to gather information about program activities, management, and implementation. Observation: Program activities were observed during the second site visits in five of the 10 research cities. Several key dimensions of program quality were examined in these observations: youth's relationships with adults and peers, structure and management of activities, and opportunities for youth decision making. A total of 23 activities were observed, most of which lasted between 30 and 60 minutes. Secondary Source/Data Review: Enrollment and participation data were collected in five of the research cities, chosen to be intensive research cities. Surveys/Questionnaires: To learn about the range of programming experiences in all cities, program directors and school coordinators in all 17 cities were asked to fill out annual organizational surveys, which gathered information about program characteristics, including programming, staff, cost, and financing data (although few cities provided information on costs). Approximately three-quarters (n = 45) of school coordinators responded to the survey. In addition, in the five intensive research cities, parents filled out a brief questionnaire when they enrolled their child. Parents provided demographic data, as well as their reasons for enrolling their children. Data from approximately 800 parents were available at the time of the analysis. |
| Data Collection Timeframe | Data were collected from late 1998 to late 1999. |
Findings:
Formative/Process Findings
| Activity Implementation | According to the evaluators, the planning period was an important step in getting programs ready to open their doors. It was a period for relationship building between partners and completion of three important tasks: choosing schools, assessing needs, and finding resources (financial and other). Even though the four models emphasize different goals, the programming on the ground was remarkably similar their first year. They all provided a strong and consistent commitment to academic work. Evaluators speculate that this occurs because of the relative ease with which these programs, particularly homework help, can be set up, and because most partners (including parents) agree that academic support is of primary importance. During the first school year, approximately 40% of activity hours were academic, 20% were cultural or creative enrichment activities, 20% were athletic, and the remainders were various. Summer program offerings were evenly split among these four areas, reflecting the reduced urgency of academic pursuits during the summer and youth's desire to be more physically active. Positive adult/youth relationships was consistently the strongest of the activity quality areas observed. Staff worked hard to make time with the youth both fun and meaningful. Low student-to-staff ratios promoted highly valued one-to-one interaction. Effective structure and management of the youth participants involved setting up age-appropriate behavioral demands that were clear to the participants and struck a balance between being firm yet warm. With middle school and high school students this often meant being flexible and somewhat hands-off. However, highly structured activities for older youth “worked” if the youth adopted them as their own. For younger children, good management required more structure. Staff were seen as needing to keep control and manage disputes, give them ideas, provide resources, and help them when they “got stuck.” Youth decision-making opportunities were not consistently observed in activities because, according to the evaluators, it requires planning and often staff training. When such opportunities were seen, they involved youth helping plan an activity or helping decide how to carry out an activity. Youth leadership was infrequently observed and typically involved situations where youth acted as group leaders or captains, tutored each other, or informally made group decisions. Middle and high school students had more formal opportunities for leadership than elementary students. |
| Costs/Revenues | Assembling resources to start and sustain programs was a major challenge for collaborative members. In general, programs assembled the needed resources through a combination of commitments from community organizations; in-kind donations by citizens and businesses; the use of the school building for program activities; grants from foundations; and donated or subsidized time from community leaders and school personnel. Generally, programs used their monetary resources to fund core staff's time, to subsidize or fully pay for some youth activities, and to cover some administrative costs. Costs covered with redirected funds or in-kind donations exceeded the expenses paid for by cash contributions by 100–200%. A review of the expenditures of a sample of 11 ESS cities showed that a school's program could be started on less than $30,000 cash annually, provided it could leverage other resources and the use of a school building. In almost all ESS sites, sufficient funding was not allotted for after hours busing services, which was found to entail unexpectedly high costs. |
| Program Context/Infrastructure | Three types of school-level governance were tried: lead-agency led, small-team governed, and shared. While the shared structure was effective in generating community interest in the initiative, it proved difficult to sustain as the initiative progressed. Almost all the schools with this governance structure shifted to more heavy reliance on either a lead-agency-led structure or a small-team structure. Lead-agency and small-team governance proved to be effective for decision making at the school level. City-level oversight committees provided important human resources to the initiative. These committees usually include very senior staff and high-level political personnel that could often find solutions to specific problems that the school level governance group could not handle. The results of the needs assessments and community mapping undertaken by the cities were crucial to building support for the initiative, especially among school principals, and to help the program design, but only if the needs of each school was assessed separately. Program planners and stakeholders benefited from the opportunity to visit active programs in other cities and this helped to transmit a concrete vision of how after school programs operate. The transition from planning to implementation was challenging in many sites. Shifts in decision making and management authority from the planning team to school-level government structures and CBOs were particularly difficult. In general, two major factors contributed to the challenges of transferring authority: first, some school-level governance structures were unprepared to take on key decision-making tasks surrounding the ESS program at their school; and second, local governance teams and agency personnel sometimes lacked clarity about their respective roles and responsibilities. |
| Program-School Linkages | Schools were mostly not the underused resources policymakers thought they were. Programs often had to compete for school space, especially gyms and computer labs, with activities being run by teachers, students, sports teams, and outside organizations. Tight school budgets and scarce custodial resources meant that school administrations' tended to limit access to even the free space, when the upkeep of the space or the cost of potential damage was high (such as with computer labs). Access was improved as the program staff built trusting relationships with key school staff. Small-team governance, where principals are included on the team, strengthened school participation in the local initiative, which may serve to create and sustain strongly integrated school/community partnerships. A common criterion for selecting schools was the principal's support. However, a large number of schools had new principals by the time implementation began. |
| Recruitment/Participation | Across sites, enrollees were 51% female (49% male) and their ethnic and racial makeup was: 45% African American, 33% white, 19% Latino, and 3% other. In the intensive research cities, 26% of enrollees lived in single parent households, 66% received free or reduced-price lunches, and 46% lived in households with incomes of $20,000 or less. Transportation challenges impeded recruitment. Given the limited supply of buses and drivers, as well as their cost, busing after hours was scarce. Youth who could not walk home and lacked adults to pick them up often could not participate. ESS coordinators and partners consistently expressed concern about transportation challenges, especially because students who required after-hours busing were often those who could most benefit from programs. The immediacy of this issue pressed programs to try to quickly develop solutions, one of which was to obtain after-hours busing. Recruitment strategies undertaken by program staff included: making presentations in classrooms and assemblies; setting up information tables at Back-to-School and Open House nights; interacting with youth in school lunch rooms and halls; holding special events for families; inviting parents for coffee; distributing and posting fliers and brochures in multiple languages; making home visits; and enlisting partners and collaborators to advertise the program through their own networks. In addition, staff consistently reported a strong reliance on informal communication, especially with students' teachers. Since so much of the recruitment was school-based, the quality of relationships between school and program staff had a direct impact on the ability of programs to recruit students. Preliminary data suggest that while ESS programs reach thousands of children who live under disadvantaged conditions, additional effort was needed to attract disadvantaged students. According to a number of program coordinators, many early enrollees were students who seemed to be more assertive and involved in school and other activities. They also noted that parents who were most involved with their children were the ones who responded first to the enrollment opportunity. Referrals from school staff and student support teams were the most common means of recruiting more disadvantaged youth. Several sites developed targeted recruitment strategies designed to be less stigmatizing than referrals, such as holding registration sessions in low-income housing units where students lived, or associating the program with their lead agencies rather than the school to attract youth who might be wary of school. Some programs also switched from a first-come, first-serve basis to a multiple-week sign-up period to enable more children to apply. Overfilled activities were filled by randomly picking from the applicants. Two factors were central to the success of recruitment: the age of youth (younger youth were easier to recruit) and the number of other youth-serving organizations and programs in the community (the more of these programs, the more difficult recruitment was). Middle and high school students were found to be harder to recruit due to their busier schedules, increased responsibilities, and greater freedom. Despite these challenges, coordinators devised several programming ideas to attract teens and have them participate consistently. Ideas that seemed to work best included flexible open-door policies and opportunities for leadership and loosely guided autonomy. Hiring staff who were closer in age and share similar ethnic or racial backgrounds with the middle and high school students also great helped recruitment. Parents supplied the following reasons for enrolling their child in ESS: their child wanted to get involved (76%); their child can make friends and have fun (54%); it will help their child do better in school (53%); it is a safe place for their child after school (41%); their child can get to and from the program easily and safely (30%); it provides dependable after school care (21%); it provides affordable after school care (19%); and school staff suggested that their child enroll (12%). |
| Staffing/Training | Although many schools tried to use a half-time school coordinator, programs learned that this was a full-time job. Project directors (who oversaw all the school coordinators) expressed concern about both the program's quality and coordinator burnout when they were only half time. To meet implementation challenges, some programs switched school coordinators from part time to full time. When programs operated more than eight hours a day, an assistant director was needed to cover the extra time. The experience and education of program coordinators did not seem as important in their ability to operate the programs as their relationships with school personnel. Youth in focus groups reported that they liked staff presence because adults could mediate potential disputes among youth and provide ideas and resources. They also reported that they wanted adult support, but didn't want staff to be constantly “on” them. |
| Systemic Infrastructure | Partners were initially attracted to ESS for a variety of reasons: about a third saw ESS as an opportunity to strengthen school-community partnerships; another third saw it as an opportunity to expand their own programming; and the final third came simply because they were invited. Cities with a history of collaboration moved more quickly in implementing their programs and had fewer tensions than did the one city with limited prior history. Cities noted two specific areas in which technical assistance provided by national intermediaries proved extremely useful in creating their collaborations: clarifying decision making and promoting the development of positive relationships among partners. The opportunities intermediaries gave cities in the planning stage to visit other cities with active initiatives and to go to cross-site conferences proved to be a particularly useful way to illustrate the potential of ESS and generate enthusiasm for the local initiatives. The results of needs assessments and community mapping undertaken by the cities were crucial to building support for the initiative, especially among school principals. The results' usefulness in guiding program planning was less clear, especially in communities that involved multiple schools. School selection was guided by a variety of criteria ranging from the academic performance and backgrounds of the school's population, to geographic considerations, to principal support. Principal support was almost always a key consideration. When cities had multiple schools involved in their program, establishing oversight committees permitted them to more easily coordinate policies and activities across schools. |
| Lessons for Policymakers and Funders | The evaluators provide several recommendations for policymakers and funders, based on the findings of this evaluation. According to the evaluators, expectations for after school programs should be tempered by well-established knowledge about what youth programs can and cannot achieve and under what timeframe and what circumstances. Specifically, the lessons learned from this evaluation are as follows:
|