You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.

www.HFRP.org

The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.

Terms of Use ▼

Program Description

Overview The U.S. Department of Education’s (DOE) No Child Left Behind (NCLB) Act of 2001 requires low-performing schools to offer Supplemental Educational Services (SES) to students from low-income families. These services are defined to include academic instruction (e.g., tutoring, remediation) that is provided outside of the regular school day. The program’s goal is to improve youth’s academic achievement. SES providers may include nonprofit, for-profit, and faith-based organizations, charter schools, private schools, and public and private colleges and universities. Local school districts can provide services, but only if they have not been identified as low performing.
Start Date Fall 2002
Scope national
Type after school, summer/vacation, before school, weekend
Location urban, suburban, rural
Setting public school, private school, community-based organization, religious institution, private facility, recreation center
Participants elementary through high school students
Number of Sites/Grantees 1,890 providers nationwide (spring 2004)
Number Served 430,000 youth in 2004–2005
Components All youth from low-income families (e.g., those who qualify for the free or reduced price meal [FRPM] program) who attend Title I schools that are not meeting their state’s adequate yearly progress goals are eligible to receive SES or be transferred to another school. Each state is required to develop provider selection criteria, approve or deny provider applications, and provide school districts with a list of available providers in their area. Districts must inform parents of SES availability and provide them with information to select providers for their children, collect parent applications or signatures, coordinate provider activities with schools, estimate per-pupil expenditures, contract with providers to provide SES, manage provider activity, and develop specific educational goals for each student in consultation with parents and providers. Parents may select any approved provider in the area served by the district or within a reasonable distance of the district. Providers must measure and report on youth’s progress regularly to teachers and parents. Schools are not required to play a role in SES implementation. Services are required to be consistent with state academic content standards and district instruction.
Funding Level NCLB requires districts to set aside 20% of their Title I, Part A allocation for school choice-related transportation and SES. Districts must spend at least 5% of this Title I, Part A allocation on SES unless demand is less than estimated.
Funding Sources U.S. Department of Education


Evaluation

Overview The DOE commissioned a multiyear study aimed at gaining insights from the early efforts of particular states and districts that could assist others in improving SES implementation. In addition, the Civil Rights Project conducted a study to examine the ability of districts to implement the requirement that schools offer SES to students attending poorly performing schools.
Evaluators Policy Studies Associates, Inc. and the Civil Rights Project at Harvard University (now at the University of California–Los Angeles)
Evaluations Profiled Early Implementation of Supplemental Educational Services Under the No Child Left Behind Act: Year One Report

Case Studies of Supplemental Services under the No Child Left Behind Act: Findings From 2003–04

Increasing Bureaucracy or Increasing Opportunities? School District Experience With Supplemental Educational Services
Evaluations Planned Many states have commissioned evaluations.
Report Availability U.S. Department of Education, Office of the Under Secretary, Policy and Program Studies Service. (2004). Early implementation of supplemental educational services under the No Child Left Behind Act: Year one report. Washington, DC: Author. Available at: www.policystudies.com/studies/school/nclb.html

U.S. Department of Education, Office of Planning, Evaluation and Policy Development, Policy and Program Studies Service. (2005). Case studies of supplemental services under the No Child Left Behind Act: Findings from 2003–04. Washington, DC: Author. Available at: www.ed.gov/rschstat/eval/disadv/supplementalyear2/index.html

Sunderman, G. L., & Kim, J. (2004). Increasing bureaucracy or increasing opportunities? School district experience with supplemental educational services. Cambridge, MA: The Civil Rights Project at Harvard University. Available at: www.civilrightsproject.ucla.edu/research/esea/increasing_bureaucracy.pdf


Contacts

Evaluation Katrina G. Laguarda
Senior Research Associate
Policy Studies Associates, Inc.
Suite 400
1718 Connecticut Avenue, NW
Washington, DC 20009
Tel: 202-939-5321
Fax: 202-939-5732
Email: laguarda@policystudies.com

Gail L. Sunderman
Senior Research Associate
The Civil Rights Project
8370 Math Sciences
Los Angeles, CA 90034
Tel: 310-267-5562
Fax: 310-206-6293
Email: glsunderman@yahoo.com
Program Jackie Jackson
Director
Office of Elementary and Secondary Education
400 Maryland Ave., SW
Washington, DC 20202
Tel: 202-260-0826
Fax: 202-205-0310
Profile Updated May 31, 2007

Evaluation 1: Early Implementation of Supplemental Educational Services Under the No Child Left Behind Act: Year One Report


Evaluation Description

Evaluation Purpose To answer the following questions about SES in its 1st year: How are states and districts implementing SES? How do states select providers? How do districts reach out to and involve parents in SES? What SES are provided? What types of providers offer and provide SES? What are the implementation challenges and successes?
Evaluation Design Non-Experimental: A sample of 6 states and 9 school districts that seemed to be relatively far along in implementing SES and that varied in location, size, and student population were selected. Interviews were conducted with state administrators responsible for administering SES, district staff involved in SES planning/implementation, and up to 3 principals and SES providers in each district. In up to 3 schools in each district, teachers participated in focus groups, and parents of SES-eligible children participated in interviews or focus groups.
Data Collection Methods Interviews/Focus Groups: Interviews and focus groups asked about SES implementation.
Data Collection Timeframe Data were collected between January and March 2003, during the 1st school year (2002–2003) that SES were offered to students.


Findings:
Formative/Process Findings

Activity Implementation SES activities tended to range from 1 to 1.5 hours twice a week for approximately one semester, most of which focused on reading instruction, using approaches ranging from detailed diagnosis and scripted lessons to homework help; some consisted mostly of academic instruction while others included various other activities. Some providers offered snacks and allowed youth to choose between reading or math activities; several offered incentives for attendance, work completion, and engagement; and others offered “creative,” enrichment, or artistic activities.

Some parents felt that the amount of time that elapsed between SES being offered and actually provided was too long; they were concerned that their child would not receive optimal benefits.
Costs/Revenues The amount of tutoring time that SES dollars bought varied within and across districts. Providers’ hourly costs ranged from $5 to $40 for 1 hour of tutoring per youth. Eight districts determined per pupil SES expenditures, which ranged from $370 to $1,136.

Several districts expressed concern about the administrative costs of SES implementation, especially the costs of mailing parent letters.

Some districts expressed frustration at having untapped SES funds tied up as a result of low SES participation. Several administrators expected decreased SES funding the next year due to increased school choice participation as more schools were identified for improvement.

Providers often had to adapt their services to district funding allocations, which usually entailed increasing the youth-to-tutor ratio and decreasing the number of sessions provided. Several providers described adjusting their program’s structure to reduce costs.
Parent/Community Involvement Providers reported making efforts to communicate with parents, sometimes hosting family nights to discuss youth’s progress. However, providers explained that parents did not usually attend these types of events and that parents’ work schedules and family responsibilities often limited communication. Providers admitted that they were usually most in touch with parents who actively sought them out and had little or no contact with those who did not.

Providers felt that language was often a barrier to communication with parents; only 3 states required providers to communicate with parents in their native language. In several districts, administrators worried that as SES expanded into schools where lesser-spoken languages were prevalent and translators were lacking, parent communication could become more problematic.
Program Context/Infrastructure Administrators and school staff in 2 districts raised safety and liability concerns in allowing outside providers to use school facilities. One principal said that she had no way to identify tutors who came into her school. In addition, before she made an issue of it, tutors would go home at the end of sessions, leaving youth unsupervised while waiting to be picked up. Two provider executives raised concerns that providers rarely took responsibility for youth and that schools did not always have staff to ensure that youth left safely and that the building was secure.

Most providers had not yet fully defined procedures to monitor and report youth progress.

The majority of each state’s providers were private for-profit and nonprofit organizations. Public schools and districts were also well represented. All 6 states had online providers, but they represented a small number of providers. Four states had faith-based providers. Many providers had worked with school-age youth, but few had served mostly low-income youth.

Online providers targeted youth who were willing to seek out facilities with Internet access, resulting in a focus on older (at least 10 years old) and highly motivated youth. Several school staff felt that online SES were inappropriate for youth most in need of help, who often lacked the discipline for such services and needed more hands-on, one-on-one help. Online providers did not necessarily result in limited district or school involvement. For example, one online provider planned to have youth work in the school’s computer lab.

Most districts believed that they had to collect data to ensure that providers honored their contracts and pledged to have an accountability system in place the next year. One district used report card grades as a temporary standard to measure provider performance. Providers disliked this provision, feeling that reports cards were subjective and did not reflect their efforts.

Provider contracts varied widely in scope and purpose. Some laid out details on issues from parent involvement to anticipated student achievement. Others were less detailed.

Only one district seemed to be aware of the need to track SES participants and was working on developing a system to do so.
Program–School Linkages Six districts enlisted schools to help implement SES, including assistance in coordinating service delivery, providing program space, notifying parents of SES, recruiting youth, and aiding parents in distinguishing between after school options. In the other 3 districts, districts did not want to burden schools with SES, though 2 of these were ultimately forced to ask schools for help.

Most school staff recognized that schools played a major role in SES implementation. Teachers in almost every district expressed interest in helping promote SES to parents—as many explained, teachers tended to be best known to parents and found it easiest to contact them— and several teachers expressed a willingness to communicate with providers about youth’s progress. School staff in several districts, however, were uncertain of their role in SES.

Many parents said that they relied on schools to help decipher provider information and that they were strongly influenced by teachers about the need for SES and in selecting providers.

Some school staff had strong reservations about SES in at least 4 districts. Some school staff felt current faculty could do the tutoring with the extra resources or that funds would be better spent on another teacher. Many teachers took a “wait and see” attitude; some expressed neutrality or even resentment, which sometimes led to hostile relationships with providers.

At least one district had a system to manage the SES available to schools and to prevent multiple providers from vying for school space. Schools were divided into geographic zones and clusters within each zone. Parents could select any provider serving a school in their cluster, but if they chose a provider at a school their child did not attend, they had to provide transportation. This system, however, had its critics; for example, one provider complained that whenever he asked a question, the district had to check with the state, which had to check with the DOE.

Only 4 states required providers to produce evidence of service alignment with state academic standards. One state required provider alignment with school curricula but not state standards.

Though providers claimed that their curricula aligned with state and district standards, they typically seemed to make only minor changes to their existing services and gave little evidence of alignment. Some providers said that their curriculum aligned with the school curriculum or that it aimed to prepare youth for state assessments, while a few mentioned referencing standards in their work. Several providers said that their SES consisted only of homework help. Several providers administered their own assessments, which they used to gauge learning needs and to place youth in learning groups or provide them with appropriate learning materials.

In several districts, school staff pointed out that youth worked from 8 a.m. to 5 p.m. without much of a break in their academic schedule. Some teachers expressed concern that to plunge these youth into more academic tutoring after school could be detrimental. Provider efforts to communicate with teachers tended to be left to individual tutors’ judgment.

Most states required SES applicants to produce evidence of effectiveness in raising student achievement, although the type of required evidence varied. Some sought evidence of teacher quality. Others required school grades, independent test data, and referral letters from families.
Recruitment/Participation The proportion of eligible youth SES served in year 1 ranged from under 1% to 42% per district.

Some districts faced challenges in communicating with parents about SES. In some districts where many parent letters were returned due to incorrect addresses, letters were often sent home with youth, which risked inconsistent distribution. Other challenges included letters that were hard to understand, ineffective methods of communicating with parents, and that some parents were ill-equipped to negotiate the SES system.

The degree to which parents knew about SES and provider options largely hinged on school and district efforts to inform them. Some parents worked with schools to select providers, while others were simply given providers’ names. Some parents said they heard about providers and SES in the media; many parents chose a particular provider based on that provider’s TV ads. Further, language barriers kept some parents from fully understanding their SES options. Many English language learner (ELL) parents said that they only received a letter in English. Some parents were not literate in English or their native language; some also said that there was no one at the school with whom they could communicate.

Districts’ follow-up with parents who expressed interest in SES varied tremendously. Some relied on schools to follow up while others gave parents additional written information. Some parents were told that they would soon receive more forms to complete, but none had yet received the forms at the time of the site visit. One school had a large drop in the number of SES participants (from 50–60 to 15–20) since several weeks passed from the time parents were notified of eligibility to the time SES began.

For most parents, transportation was the deciding factor in choosing providers; they preferred programs that provided transportation or those offered at their child’s school.

Most parents chose SES rather than a school transfer. Parents were often uncomfortable or uninterested in sending their child to an unfamiliar neighborhood or school.

In 6 districts, SES was targeted based on poverty and achievement levels. Before determining if SES needed to be targeted, several districts considered the number of youth attending identified schools and the funds available. In at least 3 districts, all eligible youth regardless of achievement were allowed to participate due to low interest. Two districts only accepted low-performing youth, despite the fact that they likely had funds to serve many more eligible youth.

SES was more difficult to implement for middle school than elementary school students. At least 2 districts met resistance from middle school students who felt self-conscious about needing help and were embarrassed to admit being in a tutoring program to peers. While absenteeism was somewhat of a problem in elementary programs, it was a major issue in middle school. In addition, since middle school students have multiple teachers, tutors often had trouble identifying school staff who were best able to help address behavior and attendance problems. Competition from sports and other after school activities was also more pronounced in middle schools than in elementary schools, further reducing SES participation.

Districts often did not give providers sufficient notice of how many youth they were expected to serve, which often left providers scrambling to schedule, staff, and structure programs. In several districts, providers had more demand than they could staff. Also, providers said they needed enough participants for SES to be a viable business proposition; several did not offer SES due to low interest. Providers in several districts also had trouble getting final participant lists, which was particularly problematic for providers that did preliminary youth testing.

Three states required providers to produce evidence of capacity to serve special populations of youth (e.g., special education, ELL).

Several administrators expected greater SES participation the following year, due to increased word-of-mouth communication.
Satisfaction Most parents of SES-eligible children were pleased to have the opportunity to enroll their child. However, many parents did not see a difference between SES and other after school programs offered. In parents’ eyes, SES was another form of tutoring that schools already had available.
Staffing/Training Most providers offered group instruction with youth-to-tutor ratios ranging from 3:1 to 8:1.

States required providers to produce evidence that their staff were qualified (all 6 states) and had undergone background checks (4 states). However, providers had varied staffing practices: some hired only certified teachers, many of whom were or had been schoolteachers; others hired graduate students. Several principals raised concerns about providers hiring teachers from low-performing schools, and some also worried that teachers who taught all day then tutored after school might burn out. Some principals, though, preferred having their own teachers in the building after hours.

Most district administrators felt that the amount of staff time required to implement SES was vastly underestimated. Several districts said they were forced to provide additional staff out of their overhead budgets or to assign existing staff additional duties.
Systemic Infrastructure District administrators suggested that SES legislation and guidance caused confusion and consternation in targeting SES. For example, one district thought they could only serve youth who were both high poverty and low achieving.

Due to limited staff capacity, state efforts to encourage provider applications were meager. Five states posted applications on their website or placed newspaper ads. Only 1 state, with the most applicants, went beyond these measures to send districts letters encouraging them to apply or seek providers to apply; they also directly contacted many providers to encourage them to apply.

States received between 20 and 288 provider applications; half had fewer than 50. Of the applicants, 86% were accepted in 2 states, 65% in 2 states, 58% in 1 state, and 56% in 1 state.

All 6 states generally followed the requirements to establish provider selection criteria and some included additional criteria to address local needs and program quality. For example, 2 states required that providers give evidence of linkages between research and program design.

States varied in the extent to which they applied provider selection criteria. Some used reviewers and scoring rubrics to evaluate applicants. Others selected providers based on their subjective notion of whether they “appeared to be sound” or whether the state had worked with a provider before. Most states pledged to improve their criteria for the year 2 application process.

The range of approved providers did not adequately match local need. States learned that providers often could not serve rural areas, special education students, or ELL youth. In addition, since few states required providers to specify the number of contact hours and costs offered in applications, districts often found that services were too expensive.

District management and oversight of providers was limited in several districts, a situation some providers found frustrated their efforts to provide SES. Of districts with multiple providers, only one district’s Title I director said he was in frequent phone contact with providers. In other districts, providers said that they had little interaction with the district.

Entering into provider contracts was new territory for most districts and proved to be very time-consuming. Some districts turned to the state for sample contracts, but that did not necessarily solve their problems if contracts were confusing to district personnel or if they were for other purposes than SES. Several districts struggled to define contract terms.

Most states had begun considering how to monitor provider performance. Two states planned to have external organizations develop an instrument for this purpose, one of which planned to use the instrument to determine if providers would be invited to continue offering SES. The majority of states, however, had not yet developed criteria to remove providers, though most agreed that those who failed to show student achievement gains for 2 consecutive years would be removed.


Summative/Outcome Findings

Academic Although too early to report program impacts, parents in one district said that their children were reading better and/or getting better grades. In another district, several teachers commented on youth’s increased confidence.

Evaluation 2: Case Studies of Supplemental Services Under the No Child Left Behind Act: Findings From 2003–04


Evaluation Description

Evaluation Purpose To answer the following questions about SES in its 2nd year: How are states and districts implementing SES? How do states select providers? How do districts reach out to and involving parents regarding SES? What SES are provided? What types of providers offer and provide SES? What are the implementation challenges and successes?
Evaluation Design Non-Experimental: A sample of 6 states and 9 school districts were selected that seemed to be relatively far along in implementing SES and that varied in location, size, and student population, including the 5 districts in the year 1 study (see Evaluation 1) that were still required to offer SES in year 2. Interviews were conducted with state administrators responsible for administering SES, district staff involved in planning or implementing SES, and up to 3 principals and SES providers in each district. In up to 3 schools in each district, teachers participated in focus groups, and parents of SES-eligible children participated in interviews or focus groups. The following criteria were used in selecting the sample: schools with the highest number of youth receiving SES in a given district; middle and high schools; providers serving the largest number of youth, and districts that were themselves state-approved providers.
Data Collection Methods Interviews/Focus Groups: Interviews and focus groups asked about SES implementation. Since 2 states were new to the sample in year 2, interviews included specific questions about year 1.
Data Collection Timeframe Data were collected between January and April 2004, during the second school year in which SES were offered.


Findings:
Formative/Process Findings

Activity Implementation Providers typically offered 1–2 hour sessions immediately after school, 2–3 times a week. SES content and structure varied widely by provider and sometimes by tutor. Providers used a range of approaches to teach reading and math: many used diagnostic test results to define SES objectives; some included guided literature reading, discussions, and extension activities to improve oral fluency, vocabulary, and comprehension; others focused on phonics and decoding skills; still other used self-paced worksheets to strengthen basic skills. Some did not use a prescribed curriculum, instead focusing on class work or homework, organizational strategies, note taking, critical thinking skills, and test prep.

In several districts, parents returned interest forms in September or early October, but SES did not begin until November or December (or even March, in one district). Several parents felt that it was easy to lose track of what they signed up for. By contrast, parents in another district were pleased that SES programs were up and running much earlier in year 2 than in year 1.
Costs/Revenues The percent of Title I (or other) funds that districts set aside for SES ranged from 2% to 21%. In the 6 districts that set aside less than 20%, 2 districts explained that they set aside enough for all eligible students, while 4 districts explained that their set aside was based on year 1 demand.

Average SES per-pupil expenditure for the 9 districts was $1,408 in year 2. For the 5 districts in both years of the study, the average increased from $967 in year 1 to $1,280 in year 2.

Several districts expressed concern over the administrative costs of implementing SES. Although districts could use Title I funds for implementation, these costs were substantial. District administrators explained that there were costs associated with mailings to schools and parents, working with providers, and legal costs associated with writing provider contracts.

Provider costs and total contact hours varied for many reasons. Staff costs depended on youth–tutor ratios, staff qualifications, and local salary scales. Some districts let providers use school facilities for free, while others charged rent, and off-site providers also paid rent. Providers who offered transportation incurred additional costs. District policies for reimbursing providers based on attendance also affected the income generated by each slot, and thus the number of tutoring hours. Further, since districts paid providers a maximum amount equal to their per-pupil Title I allocation, low-cost providers could offer more hours than high-cost ones in the same district.
Parent/Community Involvement While some providers said that they sent reports to parents every 4–6 weeks and others reported informal communication with parents, most parents reported receiving nothing from providers about youth’s work or progress. Several parents reported that they relied on teachers to keep them informed. Others said they had no idea what their child did after school.

Five states required that providers communicate with parents in their native language.
Program Context/Infrastructure Most district administrators reported beginning SES administration by contacting all state-approved providers to determine their availability in the district, and asking them to submit information about their services offered.

Districts were adept at entering into provider contracts; many used boilerplate contracts that they created and vetted in year 1. However, the time needed to finalize contracts was substantial. For example, in 2 districts, school board involvement in the vetting process caused delays. Most districts’ provider contracts contained similar content, including specifications for the following: number of tutoring hours per youth; session duration and frequency; amount paid per youth; instruction type; provision of SES on- or off-site; academic test results; program attendance, progress reports, and evaluations; agreements on youth goals and progress; service termination; liability insurance; district responsibilities; and payment/compensation terms. Most contracts were accompanied by examples of forms that providers were to complete periodically.

Of the 24 providers in the sample, SES was provided in: schools (N = 18); off-site in their own facilities, churches, or a Boys & Girls Club (N = 4); and in homes (N = 2). Within districts, some providers served several different school sites. Other districts assigned one provider per school.

Just under half of providers offered transportation, including all providers in 2 districts.

Some providers organized their programs so that they could be attended in combination with other activities. For example, after school programs of various kinds might operate for 3–4 hours after school, with SES tutoring taking place for 1–2 of those hours.

Schools faced challenges in communicating to parents the distinction between SES and other after school programs; many school staff said they did not understand the differences themselves.
Program–School Linkages Districts often relied on schools to recruit youth. Many districts gave schools sample letters to send to families of eligible youth. In some cases, school staff called parents to encourage them to enroll. One district administrator reported that teachers were instrumental in explaining services to parents. Several teachers in this district said that many parents were persuaded to sign up because their child’s teacher was providing the SES. At least 2 districts with relatively high participation rates used school-based parent coordinators or district parent liaisons to contact parents. Schools in several districts held parent meetings on SES availability and services, and some schools hosted provider fairs for parents to hear from providers directly.

Districts increasingly relied on school staff to help coordinate SES. Some principals noted that they were required to help with SES logistics, including ensuring that providers had space and that parents turned in permission forms. Further, some districts allowed schools to define eligibility beyond low-income status. Principals relied on teachers to determine which youth most needed SES. School staff in some districts said they wanted to play a bigger role in defining eligibility.

Providers worked within schools whenever possible; youth typically attended programs in their own school. Most providers hired teachers from those same schools or from the district at large.

Some providers reported formal mechanisms to communicate with teachers. One provider sent teachers checklists to identify youth’s strengths and weaknesses. In one school, providers attended school faculty meetings. School staff who worked as SES tutors informally kept teachers informed of youth’s progress. Many teachers, however, did not know which of their students received SES or from whom, and few knew much about what their students did in SES; none received written progress reports from providers. Teachers saw this lack of communication as a missed opportunity and expressed a desire to work with tutors to address particular youth’s needs; some teachers also felt that they could help ensure that youth attended SES sessions.

Although providers maintained that their curriculum aligned with state standards, some were unable to give any evidence that they did so. Others reported that they conducted alignment studies and purchased instructional materials to ensure that state standards were covered. Still others explained that they used results of state assessments to develop individual tutoring plans. Providers in at least 3 sites used diagnostic tests based on state assessments to determine tutoring content. District administrators reported that they assumed that states assessed the alignment of provider curricula with state standards as part of their provider selection criteria.

Most providers did not attempt to coordinate their instruction with district approaches, and district administrators did little to help with this. In several districts, teachers and providers agreed that explicit coordination was less important than whether SES provided academic skills. In at least 2 sites, the district’s approach to teaching reading differed considerably from that of providers. Teachers in several districts noted that some providers’ reliance on worksheets and traditional drills ran counter to their beliefs and practices.

Administrators in at least 3 districts reported relying on principals to monitor provider performance.
Recruitment/Participation Most districts mailed parents letters about SES availability. These letters were relatively straightforward and referenced the availability of math and reading “tutorial services” or “extra help.” Two districts lapsed into legislative jargon; parents in one of these districts complained that the language was inaccessible to them. Letters were usually accompanied by materials that included descriptions of provider services, a form for parents to rank their provider choices, and provider brochures. Some districts included answers to frequently asked questions, application material checklists, and due dates.

In addition to sending materials, districts made other efforts to reach parents and reported being eager to do so. One district held provider fairs at schools, sent fliers and reminders home, and had school staff conduct home visits to encourage parents to enroll. Other strategies included newspaper announcements, TV and radio ads for provider fairs, and labeling eligibility “scholarships” to give SES a more positive spin.

Although districts generally reported successfully informing parents about SES, some school staff and providers disagreed, claiming that materials were confusing, complicated, too heavily reliant on written information, or lacked adequate information on providers. Districts also cited problems communicating to parents about SES eligibility; as one district administrator explained, it was difficult for parents to understand why their child did not qualify for SES. Other parents wondered why they received a letter when their child was doing well in school.

Three of the districts in the sample in year 1 and 2 changed their targeting strategies as a result of low year 1 participation. Districts’ year 2 experiences suggested that targeting SES was unnecessary, since so few parents enrolled their children in SES.

Several districts set attendance policies, which typically involved dropping youth from the program for specific numbers of absences and/or not reimbursing providers if youth failed to attend. Some providers complained that youth absences were out of their control.

Most districts required providers to include attendance and progress reports for each participant with their monthly invoices. District administrators said that monitoring attendance was difficult. In some districts, providers had trouble managing attendance policies in a way that ensured that they got paid for all youth and sessions. For example, one district wanted copies of participant sign-in sheets and parent signatures to confirm their child attended. Providers found that it was difficult to get parent signatures; youth often failed to give the form to their parents to sign.

In year 2, every district knew how to estimate the number of youth they could serve with SES, compared to year 1, when few had a strong grasp of their capacity to serve eligible youth.

Eight districts provided SES to 7–28% of all eligible youth, and the 9th served 86%. In 6 districts, more youth were eligible than could be served with the maximum required funding; the other 3 districts could have served all eligible youth. Only one district had participation rates that reached the maximum they could support; this district went beyond its funding capacity to serve another 126 youth. Two districts provided SES to 86% or more of youth they could serve, while the other 6 districts had participation rates between 13% and 62% of youth that they could serve. Among districts with under 50% participation, districts reported that some youth did not participate because they had sufficient reading and math achievement, family responsibilities that precluded them from staying after school, or parents who could not pick them up after school.

Parents did not enroll their children in SES for a number of reasons. Many found other after school options more accessible, convenient, or appropriate, or thought that their children got enough help from other after school programs or informal teacher tutoring; several had safety concerns about their children being at school after hours or traveling home late in the day; some simply did not know enough about SES or how to sign up; and some wanted to enroll but were told that their children were not eligible. Some district and school staff noted that parents were sometimes overwhelmed by too many provider options.

The 5 districts in the sample for both years offered local, state, and federally funded after school programs that operated in tandem with SES. By year 2, most parents knew the names of the most active SES providers and most understood that their services were distinct from schools’ other after school programs. However, parents still tended to regard SES as one of many equally attractive after school activities. Some parents reported that they signed their child up for SES in addition to other programs. Parents spoke highly of arrangements where youth could participate in both SES and in recreational after school activities on the same day.

Providers in several districts reported that as SES became established, attendance was higher and more predictable. However, most respondents observed that attendance was a problem. Teachers observed that attendance was often better when services were provided in the school. Attendance was especially a challenge at the middle and high school levels. In several middle schools, school staff estimated that fewer than half of SES participants attended on any given day. In elementary schools, attendance rates were higher, sometimes approaching 95%. To combat attendance problems, several providers offered youth incentives to attend regularly.

For some parents, who studied provider brochures, choosing a provider was fairly straightforward. Other parents reported attending open houses where providers made presentations or speaking with providers at their child’s school. In several schools, providers had a regular and visible presence, attending school events, presenting in classrooms, and talking with parents. These face-to-face contacts were crucial sources of information for some parents.

Most parents were aware of the school choice option, but few seriously considered it; some wanted their child to attend school in the community, while others were reluctant to send their child to a school with teachers they didn’t know.

Parents typically heeded teacher and principal advice when deciding to sign up for SES and choosing providers. In many schools, teachers who worked for providers were a key point of contact for SES and helped recruit parents. In at least 2 districts, schools with staff who were actively involved in recruiting parents had much higher participation rates than schools with less active staff. Many parents reported that they chose a particular provider because a teacher they knew and trusted worked for that provider or had encouraged them to do so. In other schools, the principal or the Title I parent coordinator acted as a gatekeeper, directing parents to providers based on their knowledge of providers and of student needs.

Parents tended to choose providers in their child’s school; few sent their child off-site when given an on-site choice. In a couple of districts, providers that offered SES in youth’s homes were also an attractive option for some parents. While many parents reported that transportation availability was critical in selecting a provider, in 2 districts, providers that offered transportation tended to have lower enrollments than those that did not. Other parents tended to choose programs that ran until 6 p.m., so that they could get pick up their child after work.

In explaining why they signed their child up for SES, parents said that they sought to keep their child safe and occupied after school and that any “extra learning” would benefit their child. They hoped tutoring would give their child the structure and extra attention lacking during the school day and at home. Homework help was also seen as an important feature. However, some parents questioned whether a few extra hours of tutoring each week could make a difference and a few objected to providers’ instructional approaches.

All 6 states required applicants provide evidence of their capacity to serve special populations of youth (e.g., disabled and ELL).

School and district staff reported that providers offered SES that were designed to benefit all students, including high school, English language learner (ELL), and special education. High school students had more limited choices than other youth, since fewer providers served them. No district or school staff reported problems with ELL or special education students’ SES access. Spanish-speaking parents in 2 focus groups expressed satisfaction with SES.

The total number of tutoring hours each youth received ranged widely, from 18 to 120. At a rate of 2–6 hours per week, youth tended to “cycle out” of SES after 10–20 weeks. As a result, youth often received SES for only one marking period or semester, so providers who began SES early tended to serve two cohorts of youth in one academic year.

Providers required a minimum number of youth to enroll to cover their costs, which ranged from 10 youth per site to 80. Competition for youth was intense in some schools; in one district, providers raffled off prizes to youth who signed up. In almost every district, at least one provider was unable to offer services due to low interest.
Staffing/Training Tutoring was most often provided in small groups, with tutor–youth ratios ranging from 1:3 to 1:15. More than half of providers put tutors with groups of 5 to 10 youth.

All 6 states required providers to produce evidence that their staff were qualified and had undergone background checks. Of the 24 providers, 15 reported that a teaching certificate was required. A few providers had less stringent requirements. For example, one provider hired college students who were finishing teacher certification programs.

About half of providers sought to hire tutors and site coordinators directly from schools where they provided SES. Providers, teachers, and parents pointed out that teachers could be on-site immediately after school, knew youth and their parents, and could strengthen SES–school links. Others had mixed feelings; in the words of one provider, using these teachers was complicated by the fact that “teachers can’t separate their biases and attitudes from the school day.”

Many districts reported a need for additional staff for tasks such as monitoring provider services and attendance and the fact that SES sometimes took staff from their regular duties.

Providers reported that tutors got 4–20 hours of training at the beginning of the year. Many tutors reported that some tutors skipped this training, and that they typically did not receive follow-up support during the year. One problem was that most tutors worked just a few hours a week, so some were unwilling to invest large amounts of unpaid (or even paid) time in training.

Two established national providers reported that they checked on tutors regularly and conducted formal performance evaluations. Other providers had no formal systems to supervise tutors.

Providers found it challenging to hire and retain appropriate numbers of tutors to meet fluctuating demand. This sometimes led to providers’ inability to serve youth requesting SES, letting go of staff when SES demand was low, or suspending SES if youth stopped showing up.

At least 3 districts found ways to fund an SES coordinator in their district to help improve SES management and communication between providers and the district. In addition, to help distinguish among and coordinate between the various after school activities, schools in more than one district hired after school program coordinators.
Systemic Infrastructure In year 2, states asked applicants for evidence that they met provider criteria, though some asked providers only for assurances. States also added criteria to legislative requirements. For example, one state required providers serving areas with sizable ELL populations to show experience in improving this group’s academic achievement.

In year 2, all 6 states issued approved provider lists before the start of the school year. However, at least 3 districts reported receiving lists several months after states reported publishing them and 3 states continued to review and approve applications during the year.

By year 2, all 6 states refined their provider application review process, developing evaluation criteria and rubrics to score applications and organizing committees of reviewers.

Despite states’ reports of limited attempts to encourage potential providers to apply, the number of organizations applying and getting approved grew from year 1 to 2. However, state service administrators reported concerns over the limited supply of providers in small and rural districts. In one state, many of the rural areas were only served by online providers, which was problematic for schools that lacked computers and high-speed Internet connections.

Private providers other than faith-based and online were the dominant provider types in 4 of the study states and nationally. Intermediate education agencies, school districts, and school-based providers made up 25% of providers nationwide, while in 4 of the study states, these types of providers were even more common, representing 31–46% of providers (in the remaining 2 states, they were somewhat less common). The proportion of online and faith-based providers was low in all study states and nationwide.

Most states planned to monitor and evaluate provider performance, which included visiting providers and evaluating district data. Some states worked with universities or external evaluators, while others developed surveys and evaluations themselves. States generally did not attempt to monitor the number of contact hours per child or provider costs. While states knew that providers who failed to contribute to increased academic proficiency for 2 consecutive years had to be removed from state lists, states had yet to develop criteria to evaluate this requirement, and as of year 2, few providers had been removed (and none due to poor quality). One state, an exception, developed a target for provider performance in reading. Several respondents indicated that they welcomed federal assistance in this monitoring; they asked for sample data collection instruments/templates and greater interaction with the U.S. DOE on these issues.

Although most districts required agreements for providers to follow about each SES participant’s goals and progress (developed by the district in consultation with parents and providers), few districts were able to enforce this requirement and most could not say whether these agreements were fashioned in the manner mandated under NCLB.


Summative/Outcome Findings

Academic Across districts, parents gave numerous examples of ways their children benefited from SES, including improved grades, math or reading skills, and attitudes toward school. In one district, several parents appreciated that their high-school-age children earned school credit for participation that would allow them to graduate on time. Other parents reported that they observed little benefit from SES and saw no improvement in their children’s reading and math skills.

Evaluation 3: Increasing Bureaucracy or Increasing Opportunities? School District Experience With Supplemental Educational Services


Evaluation Description

Evaluation Purpose To answer the following questions: What was the extent of participation in SES? How did districts implement SES? What are the challenges to effectively evaluating SES?
Evaluation Design Non-Experimental: The sample included 11 districts in 6 states (Arizona, California, Georgia, Illinois, New York, and Virginia) that enrolled large numbers of minority and low-income youth. In most cases, the districts’ average math and reading proficiency rates were below the state average. Six districts were among the nation’s 50 largest school districts (including the 3 largest) and the other 5 were in the “central-city” portion of their Metropolitan Statistical Area. One district in Virginia was excluded from analysis, since it was not required to offer SES in year 1. On average, 19% of each of the other 10 districts’ schools were required to offer SES, ranging from 4% to 28% per district. The vast majority of students in schools required to offer SES were non-White, especially Black and Latino. Interviews were done with district officials responsible for SES implementation, superintendents, associate superintendents, Title I program coordinators, human resource directors, and transportation coordinators.
Data Collection Methods Document Review: District documents and policies related to SES were reviewed.

Interviews/Focus Groups: Interviews asked about the 1st year of program implementation.

Secondary Source/Data Review: Data were collected from district statistics (number of schools identified for improvement and youth who requested and participated in SES), records on all public schools in each state (Title I program status, number of years in improvement, enrollment, student demographics, and achievement outcomes), and media accounts of SES.
Data Collection Timeframe Data were collected October 2002 through July 2003.


Findings:
Formative/Process Findings

Costs/Revenues School staff reported that due to low program participation, large sums of money went unspent until late in the school year. The set-aside requirements created incentives for districts to use funds in instructionally ineffective ways since they could not be reallocated until late in the year.

District officials were concerned about total program and pupil costs. The law states that districts can reduce the Title I allocation to schools required to offer SES by 15% to meet the set aside requirement. Since the SES allocation was based on the total district Title I allocation, the maximum per child expenditure for SES was greater than the per pupil allocation a school received for Title I services. Further, NCLB did not provide additional funds to administer SES.

Although districts could use their Title I administration funds, this was a reallocation of existing funds; many districts used their own resources for administration.
Program Context/Infrastructure Districts had a mix of local, national, and online providers. In addition, 7 of the 10 districts applied to provide their own SES. Districts reported a number of advantages in providing SES themselves, including lower costs (allowing them to provide more SES), easier tracking of SES, and an inside knowledge of the academic curriculum.

District officials struggled to determine SES locations and schedules. District personnel felt that if services were far from school or inconvenient, youth would not attend. If a district was a service provider, union contracts influenced how districts could develop a program, including which teachers could apply to teach in the program.
Program–School Linkages Many districts were vague about how providers would coordinate SES with schools or maintain communication with teachers. In some districts, officials thought that since providers were required to assess student needs and teach state standards, SES would necessarily align with school instruction. Some districts felt that these issues posed challenges. For example, in a district offering SES on Saturdays, officials worried these services were at odds with district and school efforts to integrate remediation and extra instruction into the school day.
Recruitment/Participation The percentage of students in each district who were eligible for SES ranged from 4% to 24%. The percentage of these eligible students who requested and received SES ranged from 0% to 18% per district.
Staffing/Training Most districts assigned program administration responsibilities to existing staff. A few districts hired additional staff to handle these duties. Districts also involved legal counsel, human resources, transportation staff, communications, and budget staff.
Systemic Infrastructure Districts struggled to implement SES since most did not receive the list of schools required to offer SES until after the year began. Also, final guidance, which included additional requirements about which schools needed to offer SES, was not released until December. The late date of this requirement meant that most districts were unprepared to begin offering SES. Nonetheless, most districts offered programs, most starting late in the year.

According to district officials, many providers were cautious about entering new markets and worried about financial loss, and therefore did not offer SES in the 1st year. Some providers made demands on districts that were difficult to meet, including guaranteed enrollment and school facilities use.

© 2016 Presidents and Fellows of Harvard College
Published by Harvard Family Research Project