You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.

www.HFRP.org

The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.

Terms of Use ▼


Program Description

Overview The After-School Corporation (TASC) works in New York City and throughout the New York State region to: (a) enhance the quality of afterschool programs by emphasizing program components associated with student success and program sustainability, and (b) increase the availability of afterschool opportunities by providing resources and strategies for establishing or expanding afterschool projects.
Start Date 1998
Scope state
Type afterschool
Location urban, suburban, rural
Setting public schools
Participants elementary through high school students
Number of Sites/Grantees From 1998 to 2008, TASC directly supported 322 afterschool programs in New York City, and helped establish additional programs in the New York State region.
Number Served From 1998 to 2008, more than 350,000 youth (300,000 in New York City and 50,000 in the New York state region)
Components TASC provides grants to nonprofit organizations that establish partnerships with individual public schools. These grants support school-based projects that aim to improve academic learning, promote healthy development, and reduce anti-social behavior. Under the TASC approach, afterschool services are provided through a partnership between a public school (known as the host school) and a local nonprofit organization with ties to the community served by the host school. All students enrolled in the host school are eligible to participate in the afterschool project, which provides services free of charge from the end of each school day to approximately 6pm in the evening. The afterschool programs are intended to supplement the learning experiences of the regular school day, and programming generally emphasizes academic enrichment, homework assistance, the arts, and recreation. The intent of this program approach is to combine the community connections, youth expertise, cultural resources, and specialized foci of selected nonprofit organizations with the academic focus, facilities, and access to students that public schools can provide.
Funding Level From 1998 to 2008, TASC raised $490 million in private and public funds, after a founding investment of $125 million. Total support and revenue for fiscal year 2009 was $16,489,506.
Funding Sources The Atlantic Philanthropies, Inc., Lois Collier, NYC Council, New York City Department of Education, New York State Education Department, New York Times Neediest Cases Fund, Open Society Institute, other public sources (including local, state, and federal programs and agencies), and other foundations, corporations, and individuals.


Evaluation

Overview To assess TASC’s effectiveness, an evaluation was conducted to answer questions about quality and scale in program implementation, program effects on participating youth, and program practices linked to their successful outcomes.
Evaluator Policy Studies Associates, Inc.
Evaluations Profiled Increasing and Improving After-School Opportunities: Evaluation Results From the TASC After-School Program’s First Year

Building Quality and Supporting Expansion of After-School Projects: Evaluation Results From the TASC After-School Program’s Second Year

Patterns of Student-Level Change Linked to TASC Participation, Based on TASC Projects in Year 2

Supporting Quality and Scale in After-School Services to Urban Youth: Evaluation of Program Implementation and Student Engagement in TASC After-School Program’s Third Year

Promoting Learning and School Attendance Through After-School Programs: Student-Level Changes in Educational Performance Across TASC’s First Three Years

Building Quality, Scale, and Effectiveness in After-School Programs: Summary Report of the TASC Evaluation

After-School Programs and High School Success: Analysis of Post-Program Educational Patterns of Former Middle-Grades TASC Participants
Evaluations Planned None
Report Availability

Fiester, L., White, R. N., Reisner, E. R., & Castle, A. M. (2000). Increasing and improving after-school opportunities: Evaluation results from the TASC after-school program’s first year. Washington, DC: Policy Studies Associates.

Reisner, E. R., White, R. N., Birmingham, J., & Welsh, M. (2001). Building quality and supporting expansion of after-school projects: Evaluation results from the TASC after-school program’s second year. Washington, DC: Policy Studies Associates.

White, R. N., Reisner, E. R., Welsh, M., & Russell, C. (2001). Patterns of student-level change linked to TASC participation, based on TASC projects in year 2. Washington, DC: Policy Studies Associates.

Reisner, E. R., Russell, C. A., Welsh, M. E., Birmingham, J., & White, R. N. (2002). Supporting quality and scale in after-school services to urban youth: Evaluation of program implementation and student engagement in TASC after-school program’s third year. Washington, DC: Policy Studies Associates. Available at: www.tascorp.org/content/document/detail/1434/

Welsh, M. E., Russell, C. A., Williams, I., Reisner, E. R., & White, R. N. (2002). Promoting learning and school attendance through after-school programs: Student-level changes in educational performance across TASC’s first three years. Washington, DC: Policy Studies Associates. Available at: www.tascorp.org/content/document/detail/1436

The After-School Corporation. (2003). The After-School Corporation fifth-year report. New York: Author. Available at: www.tascorp.org/content/document/detail/1439

Reisner, E. R., White, R. N., Russell, C. A., & Birmingham, J. (2004). Building quality, scale, and effectiveness in after-school programs: Summary report of the TASC evaluation. Washington, DC: Policy Studies Associates. Available at: www.tascorp.org/content/document/detail/2466/

Policy Studies Associates. (2004). Building quality, scale, and effectiveness in after-school programs: Supplementary papers to accompany the summary report of the TASC Evaluation. Washington, DC: Author.

Birmingham, J., & White, R. N. (2005). Promoting positive youth development for high school students after school: Services and outcomes for high school youth in TASC programs. Washington, DC: Policy Studies Associates. Available at: www.tascorp.org/content/document/detail/1447/

Russell, C. A., & Reisner, E. R. (with Johnson, J. C., Rouk, Ü., & White, R. N.). (2005). Supporting social and cognitive growth among disadvantaged middle-grades students in TASC after-school projects. Washington, DC: Policy Studies Associates. Available at: www.tascorp.org/content/document/detail/1448/

Russell, C. A., Mielke, M. B., & Johnson, J. C. (2007). After-school programs and high school success: Analysis of post-program educational patterns of former middle-grades TASC participants. Washington, DC: Policy Studies Associates. Available at: www.tascorp.org/content/document/detail/1758


Contacts

Evaluation Elizabeth Reisner
Policy Studies Associates, Inc.
1718 Connecticut Avenue, NW, Suite 400
Washington, DC 20009
Tel: 202-939-5323
Fax: 202-939-5732
Email: ereisner@policystudies.com
Program Lucy Friedman
President
The After-School Corporation
925 9th Avenue
New York, NY
Tel: 212-547-6951
Email: lfriedman@tascorp.org
Profile Updated May 9, 2011


Evaluation 1: Increasing and Improving After-School Opportunities: Evaluation Results From the TASC After-School Program’s First Year



Evaluation Description

Evaluation Purpose To describe the After-School Corporation’s After-School Program (TASC) start-up and initial operations.
Evaluation Design Non-Experimental: Survey data were collected in TASC projects funded in Rounds 1 and 2 (50 projects in total). The design also included in-depth study of program start up and initial operations of 10 TASC projects (five from each of the first two rounds of grantees). Criteria for project selection for the in-depth study included evidence of practices that showed promise in supporting TASC’s key goals or innovative approaches that showed strong potential for success, and evidence of strong implementation of a program component emphasized by TASC.
Data Collection Methods Document Review: Budget documents, staff handbook, parent outreach materials, and youth participant work were reviewed during site visits to in-depth study sites.

Interviews/Focus Groups: Interviews of site coordinators, principals, project staff, teachers and other school staff, parents, and youth participants, were conducted during site visits to in-depth study sites. Data collected were similar to those collected in surveys, but sought more depth in areas of particular interest.

Observation: Project activities were observed during site visits to in-depth study sites.

Secondary Source/Data Review: Youth participant data, including demographic characteristics, school attendance, grade promotion, standardized test performance in core subjects, and enrollment and attendance in TASC projects were collected for youth participating in TASC-funded projects in Rounds 1 and 2 and nonparticipants in the host schools. TASC attendance and enrollment data were also collected for all youth participants.

Surveys/Questionnaires: Surveys were administered to site coordinators, principals, project staff, youth (in Grades 4–12), and parents (who had one or more children currently enrolled in the after school program) participating in projects funded in Rounds 1 and 2. Survey questions addressed issues of program implementation and quality, including goals, activities, opportunities, services, curriculum and instruction, project climate, links with the regular school day, governance and administration, staffing and professional development, and resources and sustainability. In addition, the survey collected data concerning youth’s program experiences and outcomes, including issues of cognitive/academic growth, growth related to special areas of program focus, college and career preparation, health and safety, social development, and institutional and community outcomes (safety/sense of safety and institutionalization of after school programming). Surveys of parents also collected information about parents’ satisfaction with the program, involvement with the project and the school, and the program’s effect on employment.

Tests/Assessments: Test score data collected included the Language Assessment Battery administered in each grade, the Regents Sequence 1 and Sequence 3 exams in mathematics administered to high school students, and the Regents English exam administered to high school students, the CTB math and reading tests administered in Grades 3–8, the Performance Assessment in Mathematics (PAM) tests administered in Grades 5–7, the Performance Assessment in Language (PAL) test administered in sixth grade, and the Regents Sequence 2 exam in mathematics administered to high school students.
Data Collection Timeframe Data were collected in spring 1999.


Findings:
Formative/Process Findings

Activity Implementation Projects’ areas of service in Year 1 clustered around (listed in order of frequency) arts enrichment, recreation, literacy and language arts, sports, cultural awareness, technology literacy, mathematics and science, community service, career preparation, and college preparation.

Over 90% of projects reported the following activities were offered 2 or more days per week: homework help/tutoring, sports/fitness, arts education, and academic enrichment.

Projects typically scheduled activities so that every child participated in two or three activities per day, often rotating among five or six activities per child throughout the week.

Because homework assistance was important to many stakeholders, it was an essential component for virtually all Year 1 projects.

Data from in-depth study projects indicated that projects embedded academic learning in ostensibly nonacademic activities.

In addition to academic learning, the after school projects offered opportunities for social development and support services and most sites offered occasional activities related to health education and healthy living.

Instructional strategies included projects that enabled hands-on discovery by children, team teaching, student engagement in integrated, culminating experiences such as performances, and individualized teaching geared to each child’s strengths and weaknesses.
Cost/Revenues TASC spent $9.2 million on a combination of startup and operating grants in Year 1. After TASC, the largest source of project resources was the Board of Education of the City of New York, which provided facilities, goods (food and supplies), and services (transportation for field trips and security) to all projects. The third largest source of project support was the grantee organizations themselves, which provided goods and services to their project sites.

The largest expense across projects was personnel; projects used about three quarters of their grant funds to cover personnel costs. The second largest expense was project administration.

Projects generally met TASC guidelines for spending approximately $1,000 per child.
Parent/Community Involvement Site coordinators’ survey responses suggested that contact with parents was regular, frequent, and conducted through varied strategies; however, interviews with site coordinators indicate that parental contact in Year 1 consisted primarily of one-time or occasional activities that occurred on an informal basis.

In order of frequency, interviews with site coordinators revealed the following efforts to involve families: recruitment of parents to organize year-end culminating events and family nights; hiring of parents as assistants or counselors; hosting special events for families; sending home flyers, newsletters, and monthly calendars; holding monthly workshops for parents; designating the school parent–teacher association president to oversee family outreach efforts for the program; referring parents to other services offered by their sponsoring agency; recruiting parents as program volunteers; and including parents on project leadership committees.

About two thirds of site coordinators reported sending materials home a few times a month. Staff reported doing this less often, with 43% sending home materials rarely or never.

More than half of site coordinators reported holding meetings or events to which parents were invited at least a few times a month. Staff reported engaging in this activity less often, with 66% reporting rarely or never holding such meetings or events.

More than half of site coordinators reported that they spent time talking on the phone with parents daily. More than two thirds of staff reported doing this rarely or never.

Almost two thirds of site coordinators reported holding informal conversations with parents on-site most days. Project staff reported that this was the most common form of communication with parents. Thirty-two percent of staff said that they held such conversations daily, 21% 1–2 times weekly, 25% a few times a month, and 22% rarely or never.

Thirteen percent of site coordinators reported holding conferences with one or more parents almost daily, while another approximately 20% held them once or twice weekly. About 20% of site coordinators and 76% of staff reported rarely or never holding conferences with parents.

Interviews with site coordinators revealed that they perceived the following factors to limit outreach to families: lack of engagement by parents of adolescents, lack of support from the host school, parents’ work schedules, parents’ language and cultural barriers, and families’ residence outside the school neighborhood.

Almost two thirds of parents said that their contact with their children’s school had increased as a result of the after school program.

Year 1 data indicated that outreach to the community occurred on a much more sporadic basis than outreach to parents.
Program Context/ Infrastructure Although stakeholders and participants in the initiative generally agreed on program goals, interviews and survey responses revealed relevant variation in individual objectives. In general, projects attempted to accommodate all of these goals, although with varying emphases across projects.

Central partners in the TASC-supported after school program included the nonprofit grantee organizations that administer the after school projects, the Board of Education (BOE), and the schools in which the projects operate. As a program partner, BOE exerted a powerful (though indirect) influence through its standards for student academic performance.

A comparison of the schools that host TASC after school projects with all New York City public schools revealed that the TASC host schools served students who were typically poorer, more likely to be non-White, and slightly more likely to be English language learners than the average student in the city’s public schools.

The vast majority of schools participating in TASC also hosted one or more other after school projects, although most of these operated for considerably less time, served fewer students, and provided more specialized services than the TASC-supported projects.

Although TASC after school activities occurred throughout the host schools in Year 1, projects did not always have access to all of the space they needed.

Most TASC projects were located in neighborhoods with high crime rates.

Seventy-one percent of youth participants said that it was true or very true that they felt safer after school as a result of the TASC program, while 96% of parents reported that the program seemed safe.

Sixty-three percent of secondary school students reported feeling safer at the after school program than at other locations where they spent time; 5% felt less safe at the program.
Program–School Linkages Although survey data demonstrated that after school staff and site coordinators communicated frequently with school staff on an informal basis, interviews suggested that these contacts involved only a handful of school staff. Few projects had formal mechanisms for cultivating relationships between the two staffs.

Virtually all site coordinators emphasized the key role that principals played in governing after school projects. Principals’ views of coordination and collaboration with the after school program were positive. Site coordinators and project staff rated coordination as generally high.

Since BOE schools were subject to stringent language and literacy student performance standards, most principals were eager to link their schools to after school programs that provide opportunities for students to read, write, and develop critical skills related to achieving the standards.

Surveys revealed that gaining access to teachers’ classrooms for after school activities and gaining access to the host schools’ computer and technology equipment were major challenges for TASC programs.

Almost three quarters of site coordinators agreed or strongly agreed with the statement that TASC’s curriculum and instruction reinforce concepts taught during the school day.
Recruitment/ Participation Parents’ survey responses revealed that the most common reasons they sent their children to the TASC program were as follows: to have a safe place for children to be after school (86% of parents), to have children do better in school (84%), to have children receive help in reading and math (83%), and to have children receive homework help (75%).

The racial/ethnic background of TASC participants reflected that of students in the host schools overall, 48% were Hispanic and 40% were African American.

A slightly higher proportion of TASC participants were designated bilingual (16%) compared to the bilingual population among nonparticipating students in the host schools (14%).

The proportion of participating students who qualified for free or reduced-price lunch (88%) was equal to the percent of qualifying nonparticipants in the host schools.

The distribution of students by gender was more even among TASC participants (50% male, 50% female) than among nonparticipating students in the host schools (48% female, 52% male).

TASC participants had a higher average school attendance rate during the school year prior to the TASC program (93%) than did nonparticipants in the host schools (83%).

Students in TASC reported engaging in several risk behaviors, the most prevalent of which were fighting, alcohol use, and sexual intercourse, in that order.

Among students for whom the BOE records include information on grade-level promotion and retention, more TASC participants (90%) than nonparticipants in host schools (84%) were promoted to the next grade in 1998.

TASC participants scored higher on several standardized tests in 1997–1998 than did nonparticipants, including the Language Assessment Battery administered in each grade, the Regents Sequence 1 and Sequence 3 exams in mathematics administered to high school students, and the Regents English exam administered to high school students. The performance of participants and nonparticipants was nearly identical on all other exams for which the evaluation has information, including the CTB math and reading tests administered in Grades 3–8, the PAM mathematics tests administered in Grades 5–7, the PAL Language test administered in sixth grade, and the Regents Sequence 2 exam in mathematics administered to high school students.

Most students continued to participate in the program for several months after enrolling. The overall daily attendance rate across all projects—the ratio of the number of days attended by students to the number of days of service—was 74%.

The proportion of active participants (defined as those participants who attended projects 3 or more program days per week) was higher in projects serving elementary schools (81% of enrolled students) than in combined elementary–middle-grades schools or middle schools (74% and 40%, respectively).

Over 22% of children in the 50 host schools were enrolled in a TASC after school program.
Satisfaction On surveys, youth participants reported a fairly high degree of satisfaction with the program. Most participants said that they liked the program, that the program engendered a sense of belonging (in terms of having friends in the project and feeling successful or comfortable in the project), and that staff conveyed a sense of support (i.e., staff care, take time to explain things, act fairly toward students).The biggest problem reported was troublemaking by a few participants.

Parents’ survey responses were generally positive about the program.
Staffing/Training In survey responses, both site coordinators and project staff indicated relatively high levels of qualifications and experience. Most site coordinators had earned at least a master’s degree and two thirds have prior experience as a manager of administrator of a social service, education, or community development program. Almost half had prior experience as a classroom teacher. Project staff had a more diverse range of qualifications and experiences, consistent with their varied roles within projects.

The racial/ethnic group most heavily represented among project staff was African American, with smaller percentages of Hispanic and White staff.

About 60% of all staff worked 15 hours a week (3 hours a day, 5 days a week); most of the remaining staff worked fewer hours.

The median student–staff ratio was near 10:1, which was the target level set by TASC.

On staff surveys, just two thirds of responding staff said they planned to return to the program in Year 2, with most of the remaining staff uncertain of their future plans. Almost all responding staff said they enjoyed working in the after school project and found the work challenging and rewarding. The dimensions of job satisfaction that received the fewest positive ratings were access to needed materials and equipment, existence of an effective policy on student behavior problems, and staff involvement in decision making.

More than half of site coordinators indicated that recruiting and hiring staff was a challenge in Year 1, especially due to the mandated 10:1 ratio and the budgetary constraints that programs’ spending cannot exceed $1,000/child. The challenge of finding qualified staff consumed much of the site coordinators’ time.

More than half of site coordinators reported holding staff meetings at least monthly while 82% of staff who worked more than 8 hours a week reported attending staff meetings weekly or biweekly.

In Year 1, PASE delivered training in three formats: weeklong training institutes, monthly meetings for site coordinators, and “core knowledge” training provided to after school project staff. Training participants favored training sessions that focused on practical, concrete topics and issues they confront in their everyday work.

More than two thirds of site coordinators said the training they received in Year 1 either completely met their needs or provided a good start and over three quarters said the quality and the content of the training and technical assistance were good or excellent and were useful.

Almost three quarters of project staff who provided survey responses said they participated in training during Year 1 and most found the training to be valuable.

Both coordinators and staff noted that more help and support would permit them to implement the ideas and strategies they learned about. In particular, site coordinators reported that they could have used more training in the following areas, especially in early training sessions: managing project staff, day-to-day operations, budgeting, developing and administering academic instruction, assessing a program’s short-term effectiveness, using computerized spreadsheets, negotiating with school partners, serving special needs students, using the BOE spending authority, and developing staff’s professional skills.
Systemic Infrastructure In Year 1, TASC established an administrative framework that governed its central operations and the operations of every funded project. This framework included both TASC’s core requirements and less formal expectations. Site-specific arrangements for internal governance relied on the administrative procedures used by the grantee organizations. Site coordinators noted in surveys that they expected a high level of oversight during the 1st year of operation, but several noted that reporting to three entities was difficult for new project leaders.

The types of grantee organizations that ran TASC sites included youth-serving organizations, community development corporations and community service organizations, neighborhood settlement houses and community centers, multigenerational family support service organizations, public and private cultural institutions, youth sports or arts organizations, faith-based organizations, and institutions of higher education.


Summative/Outcome Findings

Academic Participants in higher attendance projects reported more often than participants in other projects that they read and understood more than they did before participating in the project, finished their homework, felt safe after school, learned to speak and understand English, felt more comfortable solving math problems, and learned to work with other students.

Survey results indicate that students and parents felt that the program had improved participants’ academic skills and involvement. Seventy-seven percent of participants and 86% of parents believed that the program helped participants complete homework; 65% of participants felt the program aided them academically by enabling them to read and understand more; 62% of participants felt the program made them more comfortable solving math problems.
Community Sixty percent of secondary school students said on surveys that the program helped them feel more like part of their community and 55% said that they had learned about their community through the program.

Forty-five percent of TASC high school students reported engaging in some form of community service, one third of whom did so through the TASC program.
Family More than half of the parents who responded to surveys said that their children’s participation in the program had helped them to miss less work than before, hold onto their jobs more easily, and work more hours.
Youth Development Student survey results indicate that TASC participants felt that the program had improved their interpersonal skills (80%).

Seventy-eight percent of participants reported on surveys that the program had helped them make more friends, and 91% of parents surveyed agreed with this statement.

Survey results indicate that participants and their parents felt that TASC contributed to nonacademic learning. Participants reported in surveys that they had learned about their own and other cultures through the program (63% of participants), improved their ability to speak and understand English (52% of participants and 56% of parents), and learned to use computers for schoolwork (48% of participants).

Fifty-nine percent of secondary school students and 61% of their parents responded in surveys that the program helped participants become more comfortable speaking to groups.


Evaluations 2 and 3:
Building Quality and Supporting Expansion of After-School Projects: Evaluation Results From the TASC After-School Program’s Second Year

Patterns of Student-Level Change Linked to TASC Participation, Based on TASC Projects in Year 2



Evaluation Description

Evaluation Purpose To assess evidence of program quality in five areas: (a) youth participant recruitment, enrollment, and retention; (b) project staff recruitment, deployment, supervision, and retention; (c) establishment and maintenance of productive relationships with the host school and the community; (d) use of available resources to improve project operations and quality; and (e) selection and implementation of curricula and activities to build cognitive skills and foster participants’ personal development.
Evaluation Design Quasi-Experimental and Non-Experimental: The evaluation compared academic indicators (performance on standardized tests and school attendance) of TASC participants to nonparticipants in schools that host TASC projects. Data were collected from TASC sites funded in the first two years of TASC (Rounds 1 through 4). The in-depth study involved 15 TASC projects (five from the third round of grantees was added to the original 10 from the 1st-year study). Criteria for project selection for the in-depth study included evidence of practices that showed promise in supporting TASC’s key goals or innovative approaches that showed strong potential for success, and evidence of strong implementation of a program component emphasized by TASC.
Data Collection Methods Document Review: The following program documents were reviewed at in-depth study sites: budget, staff handbook, parent outreach materials, fiscal reports (from 73 of the 109 sites), and participants’ completed work in the after school program.

Interviews/Focus Groups: Interviews of site coordinators, principals, project staff, teachers and other school staff, parents, and students were conducted during site visits to in-depth study sites. Data collected were similar to those collected in surveys, but sought more depth in areas of particular interest.

Observation: Project activities were observed during site visits to in-depth study sites.

Secondary Source/Data Review: Participant data, including demographic characteristics, school attendance, grade promotion, standardized test performance in core subjects, and enrollment and attendance in TASC projects, were collected from the New York City Board of Education for students participating in TASC-funded projects and nonparticipants in the host schools. TASC attendance and enrollment data were also collected for all youth participants. Evaluators classified participants as “active participants” if they attended an average 60% of program days, or 3 days per week.

Surveys/Questionnaires: Surveys were administered to site coordinators, school principals, and project staff participating in projects funded in Rounds 1 through 4. Surveys were also administered to youth in Grades 4–12 and parents (who had one or more children currently enrolled in the after school program) participating in projects funded in Rounds 1 and 2 only.

Survey questions addressed issues of program implementation and quality, including goals, activities, opportunities, services, curriculum and instruction, project climate, links with the regular school day, governance and administration, staffing and professional development, and resources and sustainability. In addition, the survey collected data concerning youth’s program experiences and outcomes, including issues of cognitive/academic growth, growth related to special areas of program focus, college and career preparation, health and safety, social development, and institutional and community outcomes (safety/sense of safety and institutionalization of after school programming). Surveys of parents also collected information about parents’ satisfaction with the program, involvement with the project and the school, and the program’s effect on employment. The site coordinator survey also included questions about 42 possible activities in eight content areas (literacy skills development; cognitive development and/or academic achievement; artistic development; social and cultural awareness/exploration; health, well-being, or life skills; physical fitness—athletic or recreational; civic engagement or community service; and career exploration/development) and about the use of group projects, culminating products, and culminating performances.

Tests/Assessments: The youth participant survey included a number of assessments, measuring academic self-esteem (e.g., I think I am a good student), sense of the program as a community (e.g., people care about each other in this program), trust in and respect for teachers (e.g., the teachers in this program don’t care what I think), social interactions in the after school environment (e.g., I have a hard time finding friends here), sense of autonomy in the after school environment (e.g., I get to do what I want here), program connection (e.g., I feel like I belong here), and program opportunities to be challenged (e.g., I get to work on projects here that make me think).

Test score data collected included the Language Assessment Battery administered in each grade, the Regents Sequence 1 and Sequence 3 exams in mathematics administered to high school students, and the Regents English exam administered to high school students, the CTB math and reading tests administered in Grades 3–8, the Performance Assessment in Mathematics (PAM) tests administered in Grades 5–7, the Performance Assessment in Language (PAL) test administered in sixth grade, and the Regents Sequence 2 exam in mathematics administered to high school students. Due to the types of standardized tests administered to ninth through 12th graders, no longitudinal analysis of individual level achievement was possible.
Data Collection Timeframe Data were collected in spring 2000.


Findings:
Formative/Process Findings

Activity Implementation The primary focus areas of TASC projects identified by Year 2 coordinators were academic/cognitive/literacy development, followed by artistic development, then physical fitness/athletic development/recreation.

Year 2 opportunities for academic and cognitive development took the form of homework help, organized or recreational writing and reading activities, problem-solving games, word games, group instruction in specific academic subjects, and math games. Of these, homework help was offered most often and at the highest intensity.

TASC projects provided several types of opportunities for youth to develop their skills and exposure to the arts. These included arts and crafts activities, arts instruction, studio experiences encouraging students’ own artistic expression and practice, opportunities to develop artistic appreciation through exposure to museums and to musical and dramatic performances, and opportunities to meet practicing artists and performers.

Year 2 opportunities for social development and support services, as reported by site coordinators, included organized social events, activities to explore different cultures and languages, unstructured time for socializing, and field trips to cultural/ethnic centers, events, exhibits, and performances.

Most projects offered sports and recreation opportunities, including organized team sports instruction and activities, free time for physical play and pick-up sports, organized individual sports instruction and activities, martial arts instruction, and fitness classes and activities.

Projects also generally offered activities in which youth learned about linking their own lives to the broader world outside their school and family. These activities included service projects, discussion of current events, and career exploration.

Among the group of in-depth study projects, those in their 2nd year of operation were more likely to develop and use comprehensive program schedules than those in their 1st year. These schedules resulted in improvements in program quality.

With some exceptions, TASC projects participating in the in-depth study did not develop or use formal curricula, nor did formal lessons plans or frameworks typically guide student activities.

An important curricular approach emerging across many program sites in Year 2 was project-based and interdisciplinary learning, often combining student learning experiences in social studies, science, the arts, reading, math, and writing.
Cost/Revenues TASC continued to link the amount of project grants to each project’s adjusted enrollment figures, with the allocation formula set at about $1,000 per child. TASC grants to New York City projects totaled $18.2 million in Year 2; grants to the sites outside New York City totaled $530,000. BOE’s allocation was $2.3 million while sites reported a total of $2.3 million raised in matching funds.
Parent/Community Involvement Parent involvement at many of the in-depth study sites continued to be low; however, sites were increasingly exerting efforts to involve and inform parents.

When asked about reaching out to parents, 48% of coordinators said they had talked to at least 16 parents individually in the last month about their child’s needs or interests.

Fifty-nine percent of site coordinators reported that parents served as paid staff and 34% reported that parents served as activity assistants.

Methods for increasing parent involvement included offering adult education classes, hiring parent liaisons, and inviting parents to after school events.

The in-depth study sites revealed a range of communication strategies with parents, from a written system for documenting children’s daily experiences for parents, to a checklist on which staff indicated homework completion and any behavior problems, to no policies at all for parental communication.

Thirty-six percent of principals reported that TASC had increased parents’ attendance at parent–teacher conferences.

Ninety-four percent of parents report that the program hours met their needs.

Although site coordinators said the local community was aware of the TASC project, they reported relatively few interactions with members of the community.
Program–School Linkages Principals who were engaged in their schools’ after school projects typically offered programming suggestions, made resources available, helped remove barriers to smooth operations, and encouraged stakeholders to participate fully in the project.

Responses to principals’ surveys indicate that principals worked less on the after school projects in Year 2 than in Year 1, but visited the programs slightly more often.

In survey responses, 89% of principals agreed that there was a strong partnership between the after school program and the school, up from 84% in Year 1. On 14 out of 16 measures of school–TASC-after-school relationships, principals reported more positive responses in Year 2 than in Year 1.

There was a lack of consensus among teachers on how much integration and coordination was desirable between the school and the TASC after school project.

Second year projects that encountered difficulty in gaining access to school resources in Year 1 reported fewer difficulties in Year 2, although gaining access to computers, the library, and the computer lab remained a challenge at more than half the sites.

Principals noted that parents expressed more positive feelings about the school as a result of the TASC projects because it provided a safe place for their children after school (60% of principals agreed).
Recruitment/ Participation In Year 2, TASC projects in New York City were located in schools that served some of the city’s most disadvantaged students. TASC projects in Year 2 enrolled a total of 32,186 students, with about 64% of them enrolled in elementary schools.

The average attendance rate of TASC projects serving grades K–8 in Year 2 was 74%, with the highest rate (77%) being among elementary school children. Among 34 Year 2 projects first funded in Year 1, 17 improved their attendance rates from Year 1 to Year 2, 4 experienced no change, and the attendance of 13 declined. An average of 77% of participants met the standard of active participation each month, slightly more than in Year 1 (74%).

In Year 2, active participants were very similar to nonparticipants on measures of poverty, race/ethnicity, status as recent immigrants, status as English language learners, and academic performance.

Active participants in Grades K–12 attended school at an average rate of 92.9% while nonparticipants attended school at an average rate of 91.3%.
Satisfaction Ninety-seven percent of parents surveyed indicated that their child liked to come to the program.

Parents demonstrated a very high level of satisfaction with the TASC program. Parents were most satisfied with those programs that devote a substantial amount of time to homework completion, have clear and well communicated safety and supervision procedures, have opportunities for students to explore nonschool related interests, and provide opportunities for child socialization and recreation.
Staffing/Training Projects in Year 2 achieved overall success in recruiting, hiring, and retaining staff who were generally qualified for the role assigned to them. The evaluators estimate that TASC projects employed approximately 3,000 staff members in Year 2.

The coordinators of TASC projects typically had professional backgrounds in youth development or recreation, social service management, or education. Among other staff employed in TASC projects, about 32% reported that they were a member of the host school’s regular staff, and 55% stated they were students themselves, in most instances college students.

The racial/ethnic backgrounds of TASC coordinators/staff were non-Hispanic/Black (38%/37%); Hispanic/Latino (21%/32%); and non-Hispanic/White (37%/21%).

Staff supervision in TASC projects typically occurred in two ways: through weekly or monthly staff meetings and through site coordinators’ efforts to observe instruction and monitor lesson plans.

Site coordinators reported generally high levels of job satisfaction, with coordinators of 2nd-year projects reporting higher satisfaction levels than coordinators of new projects.

Sixty percent of the Year 2 site coordinators said that they planned to return to their jobs in the 2000–2001 school year and 19% weren’t sure.

Among project staff other than coordinators, 67% said that they planned to return to their jobs in the 2000–2001 school year and 23% weren’t sure.

In Year 2, TASC increased the variety and amount of training and technical resources available to sites and worked with service providers to target training to projects’ needs. PASE, TASC, and the organization that administers their TASC grant were the main sources of training for site coordinators and their staffs. Sixty-nine percent of coordinators described these resources as “a good start,” with fewer either saying that “they serve my purposes completely” (18%) or responding less favorably (14%). Positive responses on these survey items were higher in Year 2 than in Year 1.

The percentage of coordinators who reported that they had implemented training lessons and that the new methods had improved their projects doubled between Year 1 and Year 2 (from 21% to 43%).


Summative/Outcome Findings

Academic In grades 3–8, active participants (defined as students who attended a TASC project three or more days a week) who scored in the lowest proficiency level on citywide standardized tests of reading and math in 1998–1999 improved more than their nonparticipating counterparts. Thirty-one percent of active participants scoring in the lowest proficiency level in math in 1998–1999 scored at a higher proficiency level in 1999–2000 as compared to only 23% of nonparticipants. Further, 2% of these active participants increased their proficiency to grade level as compared to 1% of nonparticipants. These were both significant differences (p<.05). Also, a similar pattern was observed in the reading tests administered to students in grades 3–8, but the differences between nonparticipants and active participants were not statistically significant.

For eighth graders, there was a greater increase in mean scores in citywide standardized math tests from the previous year among active participants than among nonparticipants. For active participants in eighth grade, the mean gain in scale score from the previous year was 22 points or one grade level. For nonparticipating eighth graders, the increase in scale score over the same year was only 17 points.

Due to the types of standardized tests administered to ninth through twelfth graders, no longitudinal analysis of individual level achievement was possible. However, active participants were more likely to take and pass the Regents Math Sequential 1 exam by ninth grade as compared to nonparticipants: 32% of active ninth grade participants passed the exam as compared to 1% of ninth grade nonparticipants. Furthermore, of those ninth graders to pass the Regents Math Sequential 1 exam, active participants were far more likely to pass the more advanced Math Sequential 2 and 3 exams: 52% of active participants vs. In Grades 3–8, active participants who scored in the lowest proficiency level on citywide standardized tests of reading and math in 1998–1999 improved more than their nonparticipating counterparts. Thirty-one percent of active participants scoring in the lowest proficiency level in math in 1998–1999 scored at a higher proficiency level in 1999–2000 as compared to only 23% of nonparticipants. Further, 2% of these active participants increased their proficiency to grade level as compared to 1% of nonparticipants. These were both significant differences (p < .05). Also, a similar pattern was observed in the reading tests administered to students in Grades 3–8, but the differences between nonparticipants and active participants were not significant.

For eighth graders, there was a greater increase in mean scores in citywide standardized math tests from the previous year among active participants than among nonparticipants. For active participants in eighth grade, the mean gain in scale score from the previous year was 22 points or one grade level. For nonparticipating eighth graders, the increase in scale score over the same year was only 17 points.

Active participants were more likely to take and pass the Regents Math Sequential 1 exam by ninth grade as compared to nonparticipants—32% of active ninth grade participants passed the exam as compared to 1% of ninth grade nonparticipants. Furthermore, of those ninth graders to pass the Regents Math Sequential 1 exam, active participants were far more likely to pass the more advanced Math Sequential 2 and 3 exams—52% of active participants versus 15% of nonparticipants in the same grades.

A higher percentage of active participants took and passed the Regents English exam (a high school graduation requirement) than nonparticipants, 23% and 9%, respectively, by the end of the 11th grade.

Gains in reading test scores were associated with sites in their 2nd year of operation.

Gains in math test scores were associated with sites that named TASC as a primary source of training and technical assistance.

The school attendance rate of active participants in grades K–8 increased more than that of K–8 nonparticipants from 1998–1999 to the 1999–2000 school years. Over this period, the school attendance rate for active participants increased from 92.2% to 93.2%; for nonparticipants, the school attendance rate over these 2 years was steady at 90.9%. This difference in gains was statistically significant (p < .05).

For active participants, school attendance rates increased from kindergarten through fifth grade and then declined through eighth grade. In contrast, nonparticipants had attendance rates increase from kindergarten only until fourth grade and suffered a steeper decline through eighth grade. This difference was statistically significant (p < .05).

Of students with school attendance rates in the lowest quartile in 1998–1999, active participants improved their school attendance rates more than nonparticipants. Fifty-one percent of previously low-attending active participants moved out of the lowest quartile of school attendance by the end of the 2000 school year, while only 32% on nonparticipants did so. This difference was statistically significant (p < .05).

When asked whether the after school program helped them to read and understand more, 73% of elementary participants, 66% of middle school participants, and 75% of high school participants agreed.

Eighty-six percent of parents expressed agreement that the project helped their child academically.

Principals reported that the TASC project had improved the following elements of the school: overall effectiveness (84%), students’ motivation to learn (81%), students’ attitudes toward school (81%), and student attendance (77%). Ninety-four percent of responding principals said that the benefits of hosting the TASC project “very much” or “somewhat” outweighed the costs; this is higher than in Year 1.
Family Forty-five percent of principals reported that the TASC program increased parental attendance at school events by a “great extent” or “some extent.”

Parents agreed that program hours were convenient (94%), they were missing less work than before (60%), and that the program provided support to enable them to keep their job (59%) and work more hours (54%).
Youth Development Eighty-seven percent of elementary school participants reported positive interactions with other participants in the after school program.

Seventy-eight percent of participants reported a sense of belonging engendered by the program, with particularly high scores among high school participants.

Seventy-one percent of participants responded positively to questions about their exposure to engaging, stimulating activities in their TASC after school project.

Sixty-nine percent of elementary school participants responded positively to questions about their sense of the after school project as a community in which people work together. Seventy-eight percent of high school participants and 52% of middle school participants responded positively to questions about the TASC project as a community.


Evaluations 4 and 5:
Supporting Quality and Scale in After-School Services to Urban Youth: Evaluation of Program Implementation and Student Engagement in TASC After-School Program’s Third Year

Promoting Learning and School Attendance Through After-School Programs: Student-Level Changes in Educational Performance Across TASC’s First Three Years



Evaluation Description

Evaluation Purpose The third-year evaluation consisted of an implementation and an outcomes assessment.

The implementation assessment gathered evidence of program quality in seven areas: (a) participant recruitment, enrollment, and retention, (b) project staff recruitment, deployment, supervision, and retention, (c) establishment and maintenance of productive relationships with the host school and the community, (d) use of available resources to improve project operations and quality, (e) selection and implementation of curricula and activities to build cognitive skills and foster youth participants’ personal development, (f) reactions and changes in the schools hosting TASC projects, and (g) change in certain participant competencies and reactions.

The outcomes assessment gathered evidence relating to five major questions: Who participates in TASC? What are youth’s patterns of after school participation? How does TASC participation affect youth achievement overall? What types of youth derive the most academic benefit from TASC participation? How does TASC participation affect youth’s school attendance?
Evaluation Design Quasi-Experimental and Non-Experimental: Evaluators collected data on the TASC program’s 3rd year of operation in New York City sites that were first funded by TASC in either Year 1 (1998–1999) or Year 2 (1999–2000), known here as Cohort 1 and Cohort 2 projects. The evaluation administered surveys in all 96 TASC projects that were part of either Cohort 1 or Cohort 2 and were located in New York City. Youth surveys were administered in Cohort 1 projects, of which there were 44 in Year 3.

When reviewing changes in test scores and school attendance associated with participation in a TASC project, the evaluators established a baseline against which to measure change. The baseline selected was the participant’s test score or school attendance rate during the school year prior to a youth’s enrollment in a TASC project. In their analyses, evaluators compared these base-year scores to those achieved after 1, 2, or 3 years of participation in a TASC project. Gains in test scores and school attendance experienced by TASC participants were compared to those of youth who never participated in a TASC project. For these nonparticipants, the base year was defined either as the year prior to the start of a TASC project, if they were already attending a TASC host school, or as the year prior to enrolling in a TASC host school. Although the characteristics of TASC participants were nearly identical to those of nonparticipants, statistical adjustments were made to control for baseline differences between the two groups of youth.

Using data from the nonparticipants in all of the TASC project host schools, evaluators predicted the actual gains observed in scale scores on the New York City and New York State achievement tests in reading and mathematics. The student characteristics used to predict these gains, all as measured during the individual student’s base year, were the student’s base-year test scale score, eligibility for the free-lunch program, gender, grade level, race/ethnicity, classification as an English language learner, eligibility to receive special education services, and classification as a recent immigrant. One school characteristic was used, which was the percent of students in the school who were eligible for the free-lunch program in the student’s base year.

Evaluators calculated nonparticipants’ expected gains by summing together the effects of all the above characteristics on nonparticipants’ academic outcomes. The evaluation applied this expected gain formula to TASC participants, calculating the gain that would be expected if the student never participated in TASC. The evaluators then compared participants’ true gain (the difference in their test scores between years) to their expected gain, and the difference was assumed to be a result of participation in the TASC program.

A comparison of the characteristics of active participants (attending a TASC project over 60% of their days enrolled in the program, or 3 days a week) and nonparticipants showed few differences on key characteristics, although active participants tended to come from earlier grades than did nonparticipants.

An in-depth study involved 15 TASC projects from Cohorts 1 and 2. Criteria for project selection for the in-depth study included evidence of practices that showed promise in supporting TASC’s key goals or innovative approaches that showed strong potential for success, and evidence of strong implementation of a program component emphasized by TASC.

Cohort 1 and Cohort 2 surveys were completed by 95 site coordinators (response rate of 99%), 1,571 other TASC staff working in 88 projects, 69 principals of TASC host schools (response rate of 72%), and 2,103 participating youth in Grades 4–12 who were enrolled in 38 Cohort 1 projects (1,347 in Grades 4–5, 563 in Grades 6–8, and 193 in Grades 9–12).
Data Collection Methods Document Review: The following documents were reviewed at in-depth study sites: budget, staff handbook, parent outreach materials, and participants’ completed work from the after school program.

Interviews/Focus Groups: Interviews of site coordinators, principals, project staff, teachers and other school staff, parents, and youth participants were conducted during site visits to in-depth study sites. Data collected were similar to those collected in surveys, but sought more in-depth information in areas of particular interest. Focus groups were conducted with directors and senior staff of grantee organizations.

Observation: Project activities were observed during site visits to in-depth study sites using structured observational guides to examine program implementation.

Secondary Source/Data Review: Participant data, including demographic characteristics, school attendance, grade promotion, standardized test performance in core subjects, and enrollment and attendance in TASC projects were collected from the New York City Department of Education for students participating in TASC-funded projects and nonparticipants in the host schools. For nonparticipants and participants whose parents did not give consent for inclusion in the evaluation, all identifying information was removed from the individual-level data. TASC attendance and enrollment data were also collected for all youth participants. Evaluators classified participants as “active participants” if they attended an average 60% of program days, or 3 days per week. Evaluators classified participants as “highly active participants” if they attended an average of 80% of program days, or 4 days per week.

Surveys/Questionnaires: Surveys were administered to TASC site coordinators, other TASC staff, principals of host schools, and after school participants in Grades 4–12. Surveys asked these various stakeholders about their program perceptions and experiences.

Tests/Assessments: Evaluators collected test scores from tests administered to students in New York City. Beginning in third grade, students take achievement tests in reading and in math in the spring of each school year. The tests administered to fourth and eighth grade students are required by the state, which specifies the test to be administered and the rubrics for scoring. To monitor student performance on a regular, continuing basis, the New York City system contracts with CTB-McGraw-Hill, the publisher of the state tests, to create tests for the third, fifth, sixth, and seventh grades, which are appropriate for students’ age and years of education at each grade and are similar in form and content to the state tests. The city tests produce scores that can be aligned with and compared to the scores for the fourth and eighth grades. For high school students, evaluators also collected Regents exam score data.
Data Collection Timeframe Data were collected during the 2000–2001 school year.


Findings:
Formative/Process Findings

Activity Implementation The evaluators found that promising practices reports were an effective method of informing TASC sites about promising strategies and approaches used by their peers. TASC generates these products in two ways: through the Citigroup Success Fund for Promising Practices in After-School Programs, which helps site coordinators or project staff document their own practices, and the Resource Briefs and Tool Kits developed by Policy Studies Associates (PSA), which are written by evaluation staff based on interviews with project leaders and staff. Both products are disseminated to all TASC projects.

Academic enrichment continued to be a priority for principals and site coordinators, and after school activities reflected this focus to a greater extent than in prior years. Site coordinators identified the following primary focus areas: academic/cognitive/literacy development (91%), artistic development (75%), physical fitness/athletic development/recreation (68%), cultural awareness and exploration (23%), health/well-being/life skills (11%), civic engagement/community service (8%), and career exploration/development (3%).

Compared to Year 2, projects involved more youth more frequently in extended projects and group efforts. Observations in TASC projects suggest that these types of projects promoted active learning and positive interactions in contexts that youth enjoyed.

Almost all of the evaluation’s categories of academic and cognitive development activity were offered with higher levels of intensity (i.e., frequency, extent of involvement of all students, duration through the year) in Year 3 than in Year 2. The increased levels of intensity of the varied academic and cognitive activities suggest that projects had adopted broader activity menus and hence had created a greater likelihood that activities were being matched to youth’s preferences and needs.

Among artistic development activities, visual arts and crafts, and dance/movement were offered at the highest intensity, and the intensity of both types of activities increased over comparable levels in Year 2.

The greatest amount of after school time was spent in three program activities: homework (173 minutes per week or 20% of program time), art (158 minutes per week or 18%), and recreation and sports (114 minutes per week or 13%). Ten percent of program time (90 minutes) was attributed to club or other activities, which included board games (e.g., chess) and community service, as well as sports and arts activities that participants selected on club days.

Programs spent a substantial amount of time each week on transitional activities. They spent 97 minutes per week providing snacks to students (11% of program time), usually the first activity after the regular school day, and 112 minutes on transitions between activities and end of day dismissal (13% of program time).
Cost/Revenues Based on projects’ own spending (and excluding the costs of administration and supports provided by TASC and others), the average budgeted cost for a day of service was $6.76 per participant.

Personnel costs were the largest expense for almost every site, with projects spending an average of 85% of their funds for personnel.

Sites reported that they would raise a third of their overall budgeted spending from matching funds and offset credits.
Parent/Community Involvement According to survey data, projects achieved the highest levels of parent participation at special events hosted by the after school project, with 54% of site coordinators reporting that at least half of the parents typically attended special events.

Site coordinators reached out to parents in several ways in Year 3, including referring parents to local agencies or organizations for assistance or information (77%), sponsoring cultural or recreational events (70%), talking individually with parents about their child’s needs or interests (47%), holding meetings where representatives of local agencies presented information (36%), and offering classes in parenting (30%), English as a second language (16%), and computer skills or GED exam preparation (11%).

Parents who directly assisted the projects most often served as paid staff (at 68% of sites, up from 59% the previous year). They also were activity assistants (41%), members of the site’s advisory board (31%), tutors (31%), and language translators or interpreters (27%).

Many site coordinators (78%) reported that the surrounding neighborhood was either very aware or somewhat aware of the TASC project, and more than half (51%) said community members (other than paid staff or parents) helped with the after school program at least once per month.

Half of the site coordinators reported that after school participants attended a community event or volunteered at a community program at least once a month.
Program Context/ Infrastructure TASC projects operated in schools that served some of the most disadvantaged students in the city, as measured by indicators of poverty and achievement. Across all New York City public schools (based on data from 1999–2000, as cited in PSA’s 2002 reports), 75% of elementary school students were eligible for free or reduced-price lunch, as were 72% of middle school students and 47% of high school students. Among schools with TASC projects, the median percentage of students eligible for free or reduced-price lunch was higher: 88% for elementary and middle schools and 44% for high schools.

On the 2000 administration of the citywide CTB tests in reading, 42% of all elementary students and 37% of all middle school students scored at grade level or higher, compared with 32% and 24%, respectively, of students in schools with TASC projects. In math, 40% of elementary school students and 26% of middle school students scored at grade level or higher citywide, compared with 29% and 17%, respectively, for schools with TASC projects.

Based on citywide data from 1999–2000, 85% of all students in elementary schools were nonwhite, as were 83% of middle school students and 84% of high school students. The comparable figures for schools with TASC projects were 92% in elementary schools and 90% in middle and high schools.
Program–School Linkages TASC projects achieved fairly high degrees of partnership, coordination, and integration with their host schools. They typically solicited input from teachers and principals on youth’s academic needs, used school administrators to advise and monitor activities, coordinated homework assistance with classroom teachers, and adopted school themes for special projects.

In surveys, principals of schools hosting these projects indicated a belief that the projects did not require as much of their direct involvement as in previous years. This view was reflected in principals’ reports of the amount of time spent on work related to the project, the frequency of their visits to the project, the frequency of their meetings with the site coordinator, and other information affirming the strength of the school/after school partnerships.

Site coordinators’ representation on school-wide governing, coordinating, or advisory teams increased in Year 3, with over half of principals (54%) reporting TASC participation on such teams.

Most principals said that TASC enhanced the overall effectiveness of the school.

About a quarter of after school staff also worked in the host school as teachers, classroom aides or instructional assistants, or other staff. Most said that their dual roles benefited both the regular and after school programs.
Recruitment/ Participation Although TASC enrollment in New York City has nearly tripled since the program began, project enrollment continued to average only about one third of the children in the host schools. Factors limiting enrollment included school and classroom space, demand for services, and administrative and teaching capacity.

Over the 3 years of program operation, projects increased their inclusion of students with disabilities, especially in sites serving older students. Even so, students with disabilities were underrepresented in TASC projects, based on a comparison of enrollments of these students in TASC projects and in the host schools.

Although attendance rates remained stable in Year 3 (averaging 74% in projects serving Grades K–8), a slightly larger proportion of students (78% in Year 3 vs. 77% in Year 2) met the standard for “active participation” than before. In Year 3, 75% of enrolled students remained enrolled every month through May, compared with 80% of students the previous year.

The median program days attended by youth in each year rose from 80 days in 1998–1999 to 99 days in 1999–2000 and finally to 109 days in 2000–2001.
Satisfaction Youth participants generally reported strong connections to their after school project. Most reported positive social interactions with peers and described the project as a place where people work together. However, participants expressed somewhat less satisfaction with their after school project than in the past, when asked how much they liked the program overall.
Staffing/Training In Year 3, the total TASC project workforce in New York City included about 4,400 regular staff, 290 subcontracted staff, and 140 AmeriCorps members. In addition to these paid staff, projects reported an increasing use of volunteers.

Most site coordinators were college graduates and had at least 3 years of experience working in social services, youth services, community agencies, or educational organizations. About half had experience managing that type of organization.

Overall, site coordinators were more likely to be satisfied with their jobs than in previous years.

Fewer site coordinators than in previous years said that they intend to leave their job in the next year.

About two thirds of project staff (other than site coordinators) said that they intend to return to their after school job next year.

Staff who responded to the survey (other than site coordinators) had less education than was reported in previous years. A majority of staff were students themselves, either in college, graduate school, or high school.

Most staff (excluding site coordinators) earned less than $16 an hour.

Although site coordinators reported holding fewer all-staff meetings than in past years they met more frequently with subgroups of staff. Approximately 77% reported that they hold such meetings at least monthly.

At more than half of the sites, classroom management, youth development, academic enrichment or literacy development, and athletic instruction emerged as the most useful training topics for staff, as reported by site coordinators.


Summative/Outcome Findings

Academic All participants, regardless of how often they attended the TASC program, gained 2.8 scale score points more in math over 2 years than did similar nonparticipants (p <. 05).

Youth who participated in TASC the most consistently and for the longest period of time experienced the greatest gains. After 1 year of exposure to TASC, youth who were active participants in TASC projects in every year they participated performed similarly to nonparticipants. However, the same youth experienced gains in math scores that were significantly larger than the gains experienced by nonparticipants with similar characteristics after 2 (4 scale-score points, p < .05) and 3 years (6 scale-score points, p < .05) of exposure to TASC. Youth classified as “highly active” gained 6 scale-score points more than similar nonparticipants after only 2 years of TASC participation; this difference was significant (p < . 05).

Compared to responses in previous years, more elementary grade participants in Year 3 believed that the TASC program provided them with academic benefits (85% in Year 3 vs. 73% in Year 2).

On the citywide tests of reading and English language arts, TASC participants generally performed about the same as nonparticipants.

In general, the TASC participants who are at greatest academic risk made the largest math gains, when compared nonparticipants. Math benefits were clearly evident for youth who scored in the lowest of four proficiency levels in the year prior to TASC participation. The gains for these low-achieving youth were evident for active participants irrespective of their number of years of participation. Among youth from low-income families, the evaluation also found evidence of after school significant benefits in math after both 2 and 3 years of active participation (p < .05).

Among various subgroups examined, black students were especially likely to benefit from active participation in TASC projects, demonstrating gains in math over similar nonparticipants after 1, 2, and 3 years of active participation (p < .05). Hispanic students benefited significantly in math after 2 years of participation (p < .05).

Special education students and English language learners who participated actively in TASC projects showed significant benefits over similar nonparticipants in math after a year of participation (p < .05). (The evaluation had too few 2- and 3-year participants in these categories to permit results to be computed.)

Although TASC active participants demonstrated slightly higher baseline attendance in the year prior to TASC service than did nonparticipants (93.6% compared to 92.0%), the TASC active participants improved their attendance rates significantly more than did nonparticipants (p < .05).

TASC active participants also gained significantly more in school attendance than did nonparticipants at each grade level, with patterns in Grades 5–8 particularly noteworthy (p < .05). At those grades, TASC participation significantly moderated the attendance decline that was seen among nonparticipants. Although the attendance of nonparticipants decreased consistently between Grades 5 and 8, the attendance of active TASC participants rose at each grade level except for a decline between Grades 6 and 7 after 1 year of active participation. Even there, the attendance decline of TASC active participants was less than a third of the decline for nonparticipants.
Youth Development Many participants said that the TASC program gave them new experiences and helped them develop life skills.

In questions addressed just to middle school participants, many said that the TASC project helped them master a performance skill, such as playing a musical instrument, singing, or speaking in front of a group. Many middle school participants also said that the program afforded them opportunities to contribute to the design or operation of the program itself.


Evaluation 6:
Building Quality, Scale, and Effectiveness in After-School Programs: Summary Report of the TASC Evaluation



Evaluation Description

Evaluation Purpose To answer three central questions: Are TASC services meeting high expectations for quality? Do youth benefit from participation in TASC projects? What practices are associated with the greatest benefits for youth participants?
Evaluation Design Quasi-Experimental and Non-Experimental: The evaluation of TASC’s first four years consisted of an implementation and an outcomes assessment.

The evaluation focused on the group of TASC projects in New York City that were first funded in the first two years of TASC’s operations, 1998–1999 (Year 1) and 1999–2000 (Year 2). This set of projects included all 50 projects first funded in Year 1 and all 50 projects first funded in Year 2. Ninety-six of these projects continued operation in Year 3, and 95 were operating in Year 4. These projects offered the longest period during which evaluators could observe change in program implementation, and participation patterns, and participants’ educational performance.

Evaluators collected data through surveys, site visits, and the review of administrative records for Years 1–4. Survey data were collected from projects first funded in Year 1 and Year 2, except youth survey data, which were only collected from projects first funded in Year 1. Because the full sample size was not attained until Year 2 for the site coordinator, staff, and principal surveys, most cross-time comparisons reviewed in the report using this data span Years 2 through 4.

The evaluation compared TASC participants to nonparticipants who attended TASC host schools (defined as any student who attended a TASC host school while a project was open and who did not participate in a TASC project in any year). The final student sample for the evaluation contained information on 52,355 students who participated in TASC programming and on 90,806 students in the same schools who did not participate in TASC services. Gains in test scores and school attendance experienced by TASC participants were compared to those of youth who never participated in a TASC project. For these nonparticipants, the base year was defined either as the year prior to the start of a TASC project, if they were already attending a TASC host school, or as the year prior to enrolling in a TASC host school.

The PreK–8 youth who participated in TASC programs closely resembled the nonparticipating students in the same schools on measures of family income, gender, receipt of special education, status as English language learners and as recent immigrants, and prior educational performance. Although racial/ethnic characteristics were very similar across participants and nonparticipants, black students were more highly represented among participants (37%) than among nonparticipants (28%).

In contrast to the similarity of participants and nonparticipants in Grades K–8, measures available to the evaluation indicated that high school students who participated in their school’s TASC after school project differed rather consistently from nonparticipating students in the same high schools. In particular, in Year 4, TASC participants in Grades 9–12 demonstrated lower levels of educational risk than did nonparticipants, as demonstrated by lower self-reported levels of free- and reduced-price lunch eligibility, receipt of special education services, recent immigrant status, English Language Learner status, and baseline academic achievement and school attendance. Participants were also more likely than nonparticipants to be African American, female, and less concentrated in the lower grades of high school.

Although the characteristics of the K–8 students who participated in the TASC after school projects were nearly identical to those of nonparticipants, statistical adjustments were made to control for baseline differences between the two groups. Because of data limitations and less comparability between participants and nonparticipants at the high school level, statistical adjustments could only be made for participants in Grades PreK–8.

Using data from the nonparticipants in all of the TASC project host schools, evaluators predicted the actual gains observed in scale scores on the New York City and New York State achievement tests in reading and mathematics. The student characteristics used to predict these gains, all as measured during the individual student’s base year, were the student’s base-year test scale score, eligibility for the free-lunch program, gender, grade level, race/ethnicity, classification as an English language learner, eligibility to receive special education services, and classification as a recent immigrant. One school characteristic was used, which was the percent of students in the school who were eligible for the free-lunch program in the student’s base year.

Using the above predictions, evaluators calculated nonparticipants’ expected gains on academic outcomes. The evaluation then applied this expected gain formula to TASC participants, calculating the gain that would be expected if the student had never participated in TASC. The evaluators then compared participants’ true gain (the difference in their test scores between years) to their expected gain, and the difference was assumed to be a result of participation in the TASC program.

To examine which program-related factors were associated with positive outcomes for youth, the evaluators also took the results of the regression models described above and looked at which factors were related to greater student gains across TASC projects.

Evaluators categorized each participating youth as an active or nonactive participant each year. In Grades PreK–8, an active participant was one who attended a TASC project at least 60 days during the school year (out of the typical 160 days of project operations) and also attended at least 60% of the days that it was possible to attend, or an average of 3 days per week. For youth in Grades 9–12, TASC’s project structure and operation were premised on youth attending the project on a less frequent basis. Accordingly, evaluators established a different threshold for categorizing a student in Grades 9–12 as an active participant: a minimum of 20 days over a school year and 20% of the days that it was possible to attend, or an average of 1 day per week.

In addition to the overall sample, site-visit samples (in-depth study sites) included 10 projects in Year 1, 15 projects in Year 2, 15 projects in Year 3, 14 projects in Year 4, and 14 projects in Year 5. Sites were purposively selected to obtain diversity along dimensions including grade span, type of sponsor, and geographic location.
Data Collection Methods Document Review: The following documents were reviewed at in-depth study sites: budget, staff handbook, parent outreach materials, and participants’ completed work from the after school program.

Interviews/Focus Groups: Interviews of site coordinators, principals, project staff, teachers and other school staff, parents, and youth participants were conducted during site visits to in-depth study sites. Data collected were similar to those collected in surveys, but sought more in-depth information in areas of particular interest. Focus groups were conducted with directors and senior staff of grantee organizations.

Observation: In each year of the study, evaluators collected data through observations conducted in visits to a sample of TASC projects. In general, the evaluation used site-visit data to explore the relationships, issues, and tensions surrounding the design and implementation of TASC projects.

Secondary Source/Data Review: TASC and PSA developed a participant tracking system to register the following: the date each participant enrolled in TASC activities, the participant’s withdrawal date, and the participant’s daily after school attendance during the period between enrollment and withdrawal. In addition, the evaluators cross-referenced TASC participants with the New York City Department of Education’s data on participants’ biographical characteristics (date of birth, sex, race/ethnicity, eligibility for free or reduced-price lunch, and whether the student was a recent immigrant), their eligibility for special instructional programs serving English language learners and students with disabilities, and their educational characteristics (grade in school, years of enrollment in a particular school, annual school attendance rates, and achievement on state and citywide tests of reading and math). Data on additional educational characteristics were obtained for high school students, including students’ accumulation of high school credits and Regents test performance in English and math. Evaluators also identified students in host schools who did not participate in TASC and obtained the same information on these nonparticipants as it obtained on participants.

Surveys/Questionnaires: Surveys were administered to TASC site coordinators, other staff in TASC projects, principals of host schools, after school participants in Grades 4–12, and their parents. Surveys asked these various stakeholders about their perceptions of the program, as well as their experiences with the program.

Participant surveys were collected from 1,584 youth in Year 1 (surveys from 82% of participating sites), 1,865 youth in Year 2 (87% of participating sites), 2,103 youth in Year 3 (88% of participating sites), and 2,031 youth in Year 4 (98% of participating sites). The evaluation used separate survey forms for youth in Grades 4–5, 6–8, and 9–12. The surveys asked youth to assess the degree to which the TASC project had fostered positive social interactions, built a sense of community within their projects, given them opportunities to grow, exposed them to new experiences, engaged them in learning, and given them an overall satisfying experience.

Parent surveys were collected from 1,292 parents in Year 1 (70% of participating sites), and 1,630 parents in Year 2 (91% of participating sites). This survey contains questions about parents’ background characteristics, children’s time use prior to participating in after school programming, perceptions of and satisfaction with the after school programming, and perceptions of child and family outcomes associated with participation in after school programming.

TASC staff surveys were collected from 906 staff in Year 1 (86% of participating sites), 1,155 staff in Year 2 (87% of participating sites), 1,571 (92% of participating sites), and 1,369 staff in Year 4 (93% of participating sites). This survey contains questions pertaining to job characteristics; job satisfaction, supervision, and support; training/technical assistance opportunities; relationship with school; and background/experience.

TASC site coordinator surveys were collected from 47 site coordinators in Year 1 (94% of eligible site coordinators), 94 site coordinators in Year 2 (94%), 95 site coordinators in Year 3 (99%), and 79 site coordinators in Year 4 (83%). This survey contains questions pertaining to program goals, enrollment, program activities/schedule, student/staff interactions, staffing, supervision and support, relationship with school, parent/community outreach/involvement, and site coordinator background/experience.

Evaluators constructed an intensity index for various activities from these surveys, using a formula to incorporate site coordinators’ reports of the frequency and duration of specific activities and the proportion of students participating in the activity. Index values ranged from 4 at the highest level of frequency, duration, and coverage to 0 at the lowest level.

TASC principal surveys were collected from 34 principals in Year 1 (68% of eligible principals), 66 principals in Year 2 (66%), 69 principals in Year 3 (72%), and 65 principals in Year 4 (68%). This survey contains questions about principals’ perceptions of the after school program, its strengths and weaknesses, its relationship with the school and school day teachers, and its effectiveness in benefiting students.

Tests/Assessments: Evaluators collected test scores from tests administered to students in New York City. Beginning in third grade, students take achievement tests in reading and in math in the spring of each school year. The tests administered to fourth and eighth grade students are required by the state, which specifies the test to be administered and the rubrics for scoring. To monitor student performance on a regular, continuing basis, the New York City system contracts with CTB-McGraw-Hill, the publisher of the state tests, to create tests for the third, fifth, sixth, and seventh grades, which are appropriate for students’ age and years of education at each grade and are similar in form and content to the state tests. The city tests produce scores that can be aligned with and compared to the scores for the fourth and eighth grades. For high school students, evaluators also obtained Regents exam score data.
Data Collection Timeframe Data were collected between 1998 and 2003.


Findings:
Formative/Process Findings

Activity Implementation Focusing on the highest project averages in Year 4, three activity areas clustered together with very similar levels of intensity (2.71 to 2.79): artistic development, academic/cognitive/literacy development, and social/cultural awareness and exploration. Projects for which comparable data were available in all 3 years remained fairly stable in intensity levels. The greatest increase over time appeared in the intensity of social and cultural awareness activities and the greatest decrease in fitness/sports/recreation activities.

Analysis of the intensity of academic activities indicated that six activities were implemented with relatively high intensity (i.e., above 3.00) in Year 4: homework help, organized reading, recreational reading, math games, word games, and organized writing. A review of changes in the intensity of these and other academic/cognitive activities indicates small but noteworthy increases in organized reading and tutoring, suggesting more focused efforts to improve participants’ academic achievement. Noteworthy declines in this time period appear in organized writing, group instruction, computer training (likely due to projects’ reduced access to computers), and study skills activities.

In Year 4, almost all projects (97%) employed culminating performances, indicating a popularity that has remained steady over time. Overall, 67% of site coordinators said that most or all students in the after school project participated in activities culminating in a performance. Eighty-seven percent of projects involved students in one to four such activities a year, with the typical annual number of such activities increasing since Year 2.

In Year 4, 77% of site coordinators said that their project’s participants engaged in group activities that were tied in some way to one or more academic subject areas, that extended over a single day, and that involved research, writing, and discussion. This percent of projects involved in group work reflected an increase from Year 2, when 62% of TASC sites reported that they used group efforts. In Year 4, 24% of coordinators said that most or all of their participants had completed at least one group activity. Among this subgroup of coordinators, 63% said that most youth participated in three to four group activities during the school year.

In Year 4, 76% of site coordinators reported that their TASC project used one or more themes to link activities across different program components, a percentage that had remained stable since Year 2. Thirty-five percent of Year 4 projects said that the project’s themes were coordinated with those used by the school.

A little over a third of site coordinators (36%) reported in Year 4 that they used curricula developed outside the project or school, a small increase from Year 2.

According to principal surveys, 95% agreed that students had opportunities to participate in activities in TASC projects not available during the regular school day, while 66% agreed that students received additional opportunities to develop literacy skills.

Eighty-two percent of site coordinators said that they required activity or lesson plans from at least some staff. Two fifths (41%) of these site coordinators said that they regularly required activity plans from most or all staff. At some projects, the site coordinator kept activity plans on file, so that staff could borrow ideas from each other and ask the creator of a lesson plan for help in implementing it.
Parent/Community Involvement According to site coordinator surveys, 72% of projects offered parents opportunities to attend cultural or recreational events in the community, 37% offered parenting classes (e.g., classes to help parents learn about the school system and communicate with the school, how to help their children with schoolwork and prepare for tests, etc.), 36% offered opportunities to hear from and talk with representatives from local agencies or other organizations (e.g., health, police, employment and training programs), 24% offered classes to help parents develop their own skills (e.g., GED preparation or computer skills, etc.), 18% offered English as a second language classes, and 14% offered other events and activities. Analyses demonstrated increases in all of these activities across years. The increases in “opportunities to hear from and talk with representatives from local agencies or other organizations” and “opportunities to attend cultural and recreational events in the community” were significant (p < .05).

According to site coordinators, people in the neighborhoods surrounding their projects were generally aware of them, with 86% reporting that the surrounding neighborhood was either very aware or somewhat aware of the project.

According to interviews and observations, TASC projects’ engagement with the community was grounded in their sponsoring organizations’ ongoing community outreach and engagement. High school projects were especially likely to engage with the community through development of internships for participants.
Program Context/ Infrastructure Compared to New York City’s public schools overall, the schools in the evaluation sample enrolled higher percentages of students from low-income families, low-achieving students, and Black or Hispanic students. This difference was most pronounced for schools hosting PreK–8 projects and less pronounced for schools hosting high school projects.
Program–School Linkages Interviews revealed that after school staff, site coordinators, and principals perceived many unexpected benefits of hiring teachers from the regular school day. Interview responses indicated that these teachers seemed to bring insights into after school participants’ academic and developmental needs, and gained insight into their current and future students by meeting them in a more relaxed setting. Also, school day teachers were more likely to have access to school resources, either in their own classrooms or when using a colleague’s classroom. In addition, regular day teachers who worked in TASC often served as advocates for it, encouraging other teachers to share space or refer students.

Interviews with after school staff and site coordinators also revealed some negative consequences of hiring regular day teachers: Projects wanting to offer an experience different from the regular school day sometimes found that teachers wanted to conduct after school activities just like regular day classes, particularly when site coordinators did not have full control over which teachers they could hire; some teachers were exhausted by the end of the school day or found themselves short on planning time for the next day; regular school day teachers were typically more expensive to hire than nonteachers; teachers were unavailable during the regular school day for TASC training, and perceived their education and classroom experience as too advanced for them to benefit from TASC training.

Across the sites, site visits and interviews revealed that while the site coordinator communicated regularly with the principal, the coordinator had the leeway to hire and supervise staff as well as determine program activities and the weekly schedule. The principal typically acted as an after school advocate (with the regular day staff) and a leader in logistical brainstorming around such issues as sharing space and wrapping around non-TASC after school activities.

In surveys in Year 4, virtually all principals of host schools (97%) reported a strong partnership between the school and the after school project. This level of affirmation represented an increase from prior years, with 87% and 90% of principals reporting a strong partnership in Years 2 and 3, respectively. In addition, in Year 4, 86% of principals reported that the TASC project was aligned or coordinated with the school in some way.

Of surveyed principals in Year 4, 86% reported that the after school program staff solicited input from the principal and teachers on skills with which students needed help and incorporated these topics into after school activities, 65% reported that the after school program adopted school themes for special projects, 61% reported that the after school program used school administrators (e.g., deans, assistant principals) to advise or monitor activities, 51% reported that after school staff coordinated homework assistance with classroom teachers, and 51% reported that the coordinator of the after school program served on a school planning team. While the use of all of these strategies increased between Years 2 and 4, according to principals, the integration of school themes and the identification of skill needs increased most during the period.

According to site coordinator surveys, between Years 2 and 4, after school projects had increased access to the schools’ playground/outdoor activity areas, auditoriums, and gymnasiums, especially among elementary school projects, as well as increased access to the use of telephones, fax machines, photocopiers, and storage space amongst all projects. Elementary-grade projects had decreased access to computer labs and classroom computers across this period, although access to both rose for high school projects.

According to principal surveys, the following measures of program–school relationships showed major increases after Year 2: after school staff reaching out to teachers in the school to identify the needs of students, after school staff following through with the commitments they made to the principal and other school staff, after school staff taking care of the space the school provided the project, and students being properly supervised by after school staff. The percentage of principals’ positive responses decreased significantly on one measure: the after school project having enough capacity to serve all interested students.

In Year 4, principals reported that the following program elements needed attention: qualifications of program staff other than the site coordinator (56%), quality of homework help (52%), quality of academic enrichment activities (44%), coordination/integration with the school curriculum (41%), coordination with the school (31%), number of staff (22%), and staff turnover (20%). Interviews with principals suggested that concerns about staff qualifications may be linked to many principals’ preference that the after school project employ as many teachers from the regular school day as possible.
Recruitment/ Participation Within PreK–8 host schools, about 32% of students in each school enrolled in the TASC project. At Grades 9–12, 59% of students in each school enrolled, though this figure is higher primarily because several high schools automatically enrolled all students in the TASC program.

Overall, low-income students (as determined by eligibility for subsidized meals) constituted 91% of PreK–8 participants and 75% of high school participants. Hispanic and Black students accounted for 85% of participants at PreK–8 levels and 78% of Grade 9–12 participants.

Among the 75 projects serving Grades PreK–8 for which adequate enrollment data were available for Years 2, 3, and 4, project enrollment remained relatively stable, increasing slowly across program years. A factor promoting enrollment stability was TASC’s policy of adjusting grant amounts during the program year to align the amounts with actual project attendance. In Year 2, the average enrollment was 236 students, increasing to 278 in Year 3, an increase of 18%. In Year 4, the average enrollment in these projects increased slightly to 289, a 4% increase.
The proportion of students who participated in TASC for an entire school year increased moderately between Years 3 and 4. In Year 4, 77% of participating students continued to attend the project every month through May, compared with 74% the previous year. Among PreK–8 students who participated 1 year and could attend a 2nd year because they continued in a school that hosted a TASC project, 63% also participated a 2nd year.

In 2001–2002, 71% of PreK–8 participants in TASC projects met the criteria established for active participation. Among TASC participants in Grades PreK–8 during the 2001–2002 school year, a project attendance rate of 60% represented the 18th percentile of attendance rates among all TASC participants, and a project attendance rate of 80% represented the 40th percentile point. This means that 18% of all TASC participants at this grade span had a TASC attendance rate of 59% or lower, and 40% had an attendance rate of 79% or lower. Median TASC attendance rates for PreK–8 students increased in each of the 4 years of TASC program operation that were assessed. The midpoint on the attendance rate continuum increased from 78% in Year 1 to 80% in Year 2 to 83% in Year 3 and to 85% in Year 4.

During 2001–2002, 47% of participants in Grades 9–12 met the criteria for active participation. Among TASC participants in Grades 9–12 during the 2001–2002 school year, a project attendance rate of 20% represented the 38th percentile of attendance rates among all TASC participants, and a project attendance rate of 60% represented the 90th percentile point. Median TASC attendance rates for students in Grades 9–12 fluctuated slightly across the 4 years of TASC program operation.

The median number of days attended in 1998–1999 was 80 days, in 1999–2000 the median was 99 days, in 2000–2001 it was 109 days, and in 2001–2002 it was 107 days. This calculation excludes sites that submitted data for fewer than 8 months in a school year and the sites that operated only a half-year in 1998–1999 or 1999–2000.
Staffing/Training Of site coordinators in Year 4, 86% held at least a bachelor’s degree and 40% held at least a master’s degree. Although average education levels of site coordinators were high, they declined (but not significantly) over Years 2, 3, and 4. Evaluators note that this decline is likely due to the promotion of assistant site coordinators into lead positions. These individuals had not needed the higher education levels to be hired as assistant coordinators, but they proved themselves to be effective in subordinate positions and were eventually promoted based on their work experience and performance.

In Year 4, 85% of site coordinators reported at least 3 years of experience working in social services, youth services, community organizations, or educational organizations before working in a TASC after school project; this level remained stable over time. In the same year, 50% of site coordinators reported at least 3 years’ experience in managing social services, youth services, community organizations, or educational organizations prior to working in a TASC after school project; this level also remained fairly stable. In addition, 78% of site coordinators in Year 4 reported having at least 3 years of experience providing direct services to youth, a level that remained stable over the data collection period.

About a fifth of site coordinators (21%) reported that they held some type of state teaching certification from New York or another state. Of these site coordinators, 20% held a regular license to teach in New York City, 20% held a New York State certificate, and others held provisional certificates of various types, with some site coordinators holding more than one type of teaching certificate.

Survey results showed that 95% of site coordinators reported a high level of job satisfaction (above the midpoint on a scale measuring job satisfaction), a level that remained stable across all years of the evaluation.

Forty percent of responding staff held a 2-year college degree or higher. Nineteen percent of responding staff had not yet graduated from high school.

Over half of project staff (57%) who were not working as site coordinators were themselves enrolled as students while working in a TASC project. The part-time nature of many TASC jobs made after school employment compatible with course enrollment. Most TASC staff who were enrolled in education programs were college students.

In Year 4, 80% of staff reported experience in social services, youth services, community organizations, or education, 67% reported experience working in a school, 53% reported 3 or more years in social services or education, 43% reported 3 or more years providing direct services to children, and 18% reported having some sort of teacher certification.

Almost a third (32%) of staff reported that they worked in the host school in some capacity during the school day. Thirteen percent said that they worked as classroom teachers during the school day, 9% worked as classroom assistants, and 10% worked in other positions, such as instructional specialist, pupil support staff, and administrator.

Of after school staff who also worked in the host school during the school day, 50% strongly agreed that their relationships with some students improved because they got to work with them after school, 36% strongly agreed that they got a chance to try new activities and teaching strategies, 22% strongly agreed that they changed some activities or teaching strategies in their regular classroom, 7% strongly agreed that they worried about getting burned out on teaching, and 4% strongly agreed that they had less time for their regular teaching responsibilities.

Of responding paid staff, 95% indicated that they were “highly satisfied” with their work in TASC.

As in previous years, almost two thirds (65%) of project staff in Year 4 reported that they intended to return to their jobs in the upcoming year. Ten percent said that they would not return, and 25% said that they were not sure whether they would return. Among those not returning or not sure, the main reason was needing to work more hours or full-time.

Of site coordinators in Year 4, 87% reported that staff meetings were held at least once a month, which was an increase in reported staff meetings from prior years. In addition, 71% of site coordinators reported that they held meetings of subgroups of staff at least once a month. The typical staff-meeting agenda included discussion of new activities, upcoming staff development, policy changes (most often regarding student attendance), and the sharing of lesson plans.

When asked what types of training were most valuable, project staff identified topics that were most closely related to their core day-to-day responsibilities, especially how to involve and teach after school participants. For site coordinators, these topics centered on project management and operations. Other staff were most interested in learning how to design activities to engage, involve, and teach after school participants.

Examined across 3 years of staff responses, the percent of other after school staff who said they participated in after school training as part of their job increased steadily, from 62% to 68% to 78% in Years 2, 3, and 4, respectively. All of the sites visited in the evaluation opened their program year with a 1–2-week staff orientation that covered topics such as child-abuse detection, classroom management, and group cohesion building, as well as specific instruction in leading arts and academic enrichment activities.

Staff members who did not participate in training were mainly those who worked in the TASC project 10 or fewer hours a week (47% of all staff not participating in training), were 21 years old or younger (17%), and worked in the school during the regular day and also worked more than 10 hours a week in the after school project (7%).

The level of satisfaction with training reflected a steady upward shift, with site coordinators who reported that the training served their purposes “completely” increasing from 18% in Year 2 to 22% in Year 3 to 38% in Year 4.

Over half of site coordinators and staff reported that they had implemented ideas and strategies from the training and technical assistance and that the implementation had improved the TASC project. Another third of each group was in the process of implementing ideas and strategies gleaned from the assistance.


Summative/Outcome Findings

Academic Among all TASC PreK–8 participants, the average change in scores on the math test was 1.4 standardized scale score points more after 2 years of participation than would be predicted from students’ characteristics. The difference between this change in performance and that of nonparticipants was significant (p < .05). Participants earned 0.2 more points than similar nonparticipants after 1 year of TASC participation, which was significant (p < .05).
After 1 year of exposure to TASC, active PreK–8 participants gained more in math performance than expected based on their characteristics (p < .05). Students who were active TASC participants also experienced gains in math scores that were significantly larger than expected after 2 years of exposure to TASC (p < .05).

Active PreK–8 participants who scored at or above grade level in math (proficiency levels 3 and 4) in the year prior to enrolling in a TASC project had significantly larger gains than predicted after both 1 and 2 years of active participation. Active PreK–8 participants who scored below grade level in math (proficiency levels 1 and 2) in the year prior to enrolling in a TASC project had mixed results in math. They gained significantly less than predicted after 1 year of active participation (p < .05) but significantly more than expected after 2 years of participation (p < .05).

Active PreK–8 participants who were eligible for free lunch in the year prior to TASC participation gained more points than expected in math after both 1 and 2 years of participation. Active PreK–8 participants who were not eligible for free lunch in the year prior to TASC participation gained fewer points than expected after participating in TASC for 1 year but more points than expected after 2 years.

Active PreK–8 Black and Hispanic participants gained more points than expected in math after 1 and 2 years of exposure to TASC, while White and Asian students gained the expected number of points.

Special education PreK–8 students who attended TASC projects gained more points than expected on math tests after a year of participating in TASC, as did English Language Learners. Recent immigrants, in contrast, gained fewer points than expected in math after a year of TASC participation.

No subgroup effects were found for PreK–8 on the basis of participants’ gender.

In the four sites for which fairly complete Regents data were available, high school TASC participants passed more exams earlier in their high school career than did nonparticipants, with active participants especially likely to pass five exams by the end of 12th grade in two of the four schools.

In the sites for which sufficient data on high school credits earned were available, the average number of credits earned among high school TASC participants was significantly higher than the average for nonparticipants (p < .05). In the two sites for which data on eighth grade proficiency levels were available, 7 out of 10 comparisons showed that participants had earned more credits at the end of ninth grade than nonparticipating students with equivalent eighth grade scores.

No achievement differences were evident in ELA/reading tests.

While TASC PreK–8 participants began TASC with higher school attendance rates than nonparticipants, after 1 year of TASC exposure, the average attendance rate among all participants increased by 0.53 percentage points, compared with an increase of 0.11 percentage points for nonparticipants, for a net difference of three quarters of a day over a 181-day school year. The corresponding increase among active participants was 0.75 percentage points, yielding a net difference in gains in school attendance of 1.2 days over the school year.

After 2 years of participation, the school attendance rates for all TASC participants increased by 0.68 percentage points, compared with 0.38 percentage points for nonparticipants, the equivalent of attending an additional half-day of school per year, compared to nonparticipants. Over the same period, the school attendance rates of active participants increased by 0.80 percentage points, for an additional three quarters of a school day per year, compared to nonparticipants.

These significant attendance gains (p < .05 for each) were still found after controlling for youth’s attendance rates prior to beginning TASC. Though preexisting differences in attendance rates were greater for high school youth, TASC participants’ attendance rates declined less and sometimes increased in comparison with nonparticipants, after controlling for students’ prior level of school attendance.

The attendance gains associated with active TASC participation were largest in the middle grades. For each of Grades 5–8, the difference between participants and nonparticipants in attendance gains was positive and significant (p < .05).

Participants made more positive 1-year gains in both English language arts (ELA) and math test scores in TASC projects that offered a high intensity of academic and cognitive development activities.

Participants made more positive 1-year gains in both English language arts (ELA) and math test scores in TASC projects that offered high intensity in activities focusing on fitness, sports, and recreation.

In TASC projects in which the site coordinator had a teaching certificate, participants performed better in both English language arts (ELA) and math test scores than participants in other sites.

In TASC projects where the site coordinator required project staff to submit lesson or activity plans, participants made greater gains on ELA and math tests than in programs where the site coordinator did not require lesson plans.

In sites where at least 25% of project staff had a 4-year college degree, participants had more positive changes in math and ELA test scores than in TASC sites with a lower proportion of school staff members with such degrees.

Positive relationships with 1-year student gains in reading and math were found in programs that employed staff who worked in the host school in some capacity (with associations found when more than 25% of staff reported this characteristic), spoke a language in addition to English (when more than 50% of staff reported this characteristic), and were under the age of 35 (when more than 75% of staff reported this characteristic).

Qualitative evidence suggests that certain program features were especially important to youth outcomes, including (a) the partnership between nonprofit sponsoring organizations and host schools, (b) the location of after school services within participants’ own schools, (c) the employment of a full-time project site coordinator, and (d) the expectation that participants would attend the after school project almost every day.

The following percentages of principals reported that TASC “very much” enhanced students’ attitudes toward school (56%), improved student attendance (42%), enhanced students’ motivation to learn (34%), contributed to improved student skills in math (24%), and contributed to improved student skills in reading (23%).
Family According to principal surveys, 79% agreed that parents expressed more positive feelings about the school because it provided a safe place for their children after school.

Thirty-one percent (31%) of principals reported that TASC “very much” increased parents’ attendance at school events and 15% reported that it “very much” increased parents’ attendance at parent–teacher conferences.
Prevention Forty-one percent of principals reported that TASC “very much” improved student safety, and 17% reported that it “very much” reduced vandalism at the school.
Systemic Forty-five percent of principals reported that TASC “very much” enhanced the overall effectiveness of the school.


Evaluation 7: After-School Programs and High School Success: Analysis of Post-Program Educational Patterns of Former Middle-Grades TASC Participants



Evaluation Description

Evaluation Purpose To examine whether TASC participation in the middle grades (grades 6–8) promotes the development of protective factors that result in higher levels of school engagement and academic progress and in lower levels of delinquent behavior.
Evaluation Design Quasi-Experimental: High school engagement and academic progress of former middle-school TASC participants (program group) were compared with that of two matched comparison groups.  For the first comparison group, evaluators identified youth who attended a TASC host school but never participated in TASC, during the same time period as program youth. The second comparison group was selected using a two-stage matching approach and included youth who attended schools similar to the TASC host schools. High school data were obtained on youth who were enrolled in grades 6¬–8 from Fall 1998 through Spring 2002. Analysis was limited to youth for whom New York City (NYC) Department of Education (DOE) high school data were available, indicating that they remained in NYC public schools through at least grade 9.

For the program group, 2,390 youth in grades 6–8 were identified from 28 TASC programs throughout NYC who attended TASC at least 60 days and 60% of possible days in at least one year from Fall 1998 to Spring 2002. The first comparison group was matched to program youth based on key youth-level demographic variables (gender, race/ethnicity, and English Language Learner [ELL] and free and reduced price lunch [FRPL] program status), yielding 1,933 matched pairs. For the second comparison group, in the first stage, each TASC school was matched to two NYC schools that did not offer TASC based on key school-level demographic variables (ethnicity, recent immigrant levels,  and ELL and FRPL program status) and school-level performance variables (enrollment, attendance, math, and English achievement levels). In the second stage, students at matched non-TASC schools were matched to program youth based on key youth-level demographic variables (gender, race/ethnicity, and ELL and FRPL program status), yielding a sample of 2,208 matched pairs. Because middle-grades TASC participation could be expected to have an effect on measures of middle-grades educational performance, the study did not control for school achievement or attendance in the matching procedure. As a result, on average, participants had a higher eighth-grade school attendance rate than both of the matched groups (93% vs. 90% for nonparticipants at TASC schools and 93% vs. 92% for matched youth from non-TASC schools). This difference could be either a reflection of a self-selection bias, with middle-grades students who were more engaged in school also more likely to participate in an afterschool program, or the reflection of a positive effect of TASC participation on the student attitudes and behaviors that determine school attendance rates. Participants and matched nonparticipants from the same schools did not differ significantly in terms of their eighth-grade reading or math scores.
Data Collection Methods

Secondary Source/Data Review: Data were collected from the TASC evaluation database and DOE databases on all program and comparison youth on key youth-level demographic variables (as outlined above) and high school engagement and academic progress (school attendance, persistence, credit accrual, test scores, suspension rates, on-time grade promotion, and graduation). In addition, school-level demographic and performance data on students (both of which are outlined above) were collected for program schools and matched non-TASC schools. Because of data limitations, analyses of high school credits were limited to cohorts entering grade 9 in 2002 or later.

Tests/Assessments: For the cohorts expected to complete grade 12 by Spring 2005 and still enrolled in NYC schools in grade 12, passing rates on Regents exams (state-level subject-area tests) were examined to determine students’ eligibility for each of three levels of diplomas: Regents, Advanced Regents, and Local. To receive a Regents diploma in 2005, students in New York State needed to pass, with a score of 65 or higher, five Regents Exams: Integrated Algebra (or Math A), Global History and Geography, U.S. History and Government, Comprehensive English, and any one science Regents. To receive an Advanced Regents Diploma, students needed to also pass an additional Regents science exam, (Earth Science, Chemistry, or Physics), an additional math exam (Geometry, Algebra 2/Trigonometry, or Math B), and a foreign language exam. Local school districts could also choose to award a Local Diploma.

Data Collection Timeframe Data were collected from the 1998–1999 to 2004–2005 school years.


Findings:
Summative/Outcome Findings

Academic Program youth had significantly higher school attendance rates in the early high school years than either of the comparison groups (p = .00 for each), with program youth attending almost seven more grade 9 school days and almost six more grade 10 days than matched nonparticipants from TASC middle schools; and about four more grade 9 days than matched youth from non-TASC middle schools.

Significantly more program youth remained enrolled in a NYC high school for at least 2 years after grade 9 than did matched nonparticipants from TASC middle schools, although the size of the effect was relatively small (p < .05, 89% vs. 86%). There were no significant differences in enrollment between program youth and youth from non-TASC matched schools.

Program youth earned significantly more high school credits in grade 9 than did matched nonparticipants from TASC schools (10.2 credits vs. 9.7 credits, p = .02). In addition, program youth earned significantly more high school credits in grade 11 than did matched youth from non-TASC schools (p = . 02), although the grade 11 sample was small.

Although program youth were significantly more likely to be promoted to grade 10 on time than were matched nonparticipants from TASC schools (p < .05), this difference was negligible: 73% of program youth were promoted on time, compared to 69% of nonparticipants. No significant differences were found for the second comparison group of non-TASC school youth.

There were no significant differences at any grade level (9–12) in the number of Regents exams passed by program youth and either of the comparison groups.

No significant differences were found between program youth and comparison youth in terms of the percent who were eligible to receive each type of high school diploma.
Prevention Both program youth and comparison youth had relatively low suspension rates, consistent with NYC district-wide patterns. Although program youth had significantly fewer suspensions in grade 9 than did matched comparison youth (p < .05), the effect sizes were negligible (4% vs. 5% for matched nonparticipants from TASC schools and 4% vs. 6% for matched youth from non-TASC schools).

© 2016 Presidents and Fellows of Harvard College
Published by Harvard Family Research Project