You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.

www.HFRP.org

The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.

Terms of Use ▼

Program Description

Overview The Transition to Success Pilot Project (TSPP) attempted to strengthen the academic and social development of at-risk youth in Boston, Massachusetts, through the provision of after school programming and targeted family outreach. These after school services were coordinated with intensive academic tutoring that children received through the Boston Public Schools' (BPS) Transition Program (TP). TP provides 40 hours of remedial tutoring services for under-performing students who do poorly on standardized tests and are therefore recommended for intensive academic support for promotion to the next grade. TSPP chose to target TP students because they represented the population of BPS students that were most academically disadvantaged and therefore might benefit from the learning support that after school programs could offer. The goal of the pilot project was to support academically at-risk students through high quality after school programs, especially because this population is generally the least likely to enroll in such programs.
Start Date fall 2001; pilot project completed in spring 2002
Scope local
Type after school
Location urban
Setting public schools, private schools, community-based organizations, religious institutions, private facilities, recreation centers
Participants elementary and middle school students (third through eighth graders)
Number of Sites/Grantees six program sites (2001–2002)
Number Served 116
Components Each TSPP program site was run by a community-based organization that partnered with the site's host school. The sites were located in diverse neighborhoods throughout Boston. To be eligible for inclusion in TSPP, programs were required to have well-established connections with their host schools, program curricula and structures that fixed learning as an intentional objective of their activities, and the organizational capacity to address the individual needs of at-risk children.

TSPP offered services as an addition to the services provided by TP. In addition to TP's after school tutoring and school-day intensified academic assistance, TSPP offered students: (1) access to existing high quality after school programs through scholarships designated for students who could not afford tuition; (2) coordinated communication among each child's teachers, after school staff, and parents, facilitated by a TSPP Program Coordinator; and (3) access to learning enrichment resources (e.g., specialized after school curricula) that programs identified as needed for the after school program. The after school programs typically combined structured homework assistance, academic-oriented and enrichment activities, and various clubs. Most programs also incorporated a range of family and support services, such as family nights, home visits, etc.

TSPP providers aggressively recruited TP students to participate through mailings, telephone calls, and home visits to parents, as well as one-on-one recruitment of students. In addition, each site hired a staff member, the Program Transition Coordinator, to coordinate communication (between the program, school, and home) and services for individual TSPP students.
Funding Level $295,000
Funding Sources City of Boston's 2:00-to-6:00 After-School Initiative; Massachusetts Department of Education; Mayor's Office of Community Partnerships (now the Boston Centers for Youth & Families)
Other TSPP was a collaboration among the Boston 2:00-to-6:00 After-School Initiative, the Boston Public Schools, and Massachusetts 2020. The Boston 2:00-to-6:00 After-School Initiative was created by Boston Mayor Thomas Menino in 1998 to support the expansion of high quality after school programs that provide learning and social development opportunities for children and youth. Massachusetts 2020 is a nonprofit foundation focused on expanding educational and economic opportunities for children and families across Massachusetts, with an initial focus on expanding after school and summer learning opportunities for children.


Evaluation

Overview The evaluation assessed the degree to which participation in full after school programming, outreach to parents, and strengthened connections between school-day teachers and the after school program enhanced the progress of the TP students.
Evaluator Center for Evaluation and Research With Children and Adolescents, of the Massachusetts Society for the Prevention of Cruelty to Children
Evaluations Profiled Research Report: The Transition to Success Pilot Project
Evaluations Planned none
Report Availability Massachusetts 2020. (2004). Research report: The Transition to Success Pilot Project. Boston, MA: Author. Available at www.mass2020.org/finaltransition.pdf (Acrobat file).


Contacts

Evaluation Dr. David Robinson
Center for Evaluation and Research With Children and Adolescents
Massachusetts Society for the Prevention of Cruelty to Children
399 Boylston Street, 3rd Floor
Boston, MA 02116
Tel: 617-587-1500
Fax: 617-587-1582
Email: drobinson@mspcc.org
Program Dr. David Farbman
Research Director
Massachusetts 2020
18 Tremont Street, Suite 1120
Boston, MA 02108
Tel: 617-723-6747
Fax: 617-723-6746
Email: david@mass2020.org
Profile Updated May 13, 2004

Evaluation: Research Report: The Transition to Success Pilot Project


Evaluation Description

Evaluation Purpose To assess the degree to which participation in TSPP after school programming, outreach to parents, and strengthened connections between school-day teachers and after school program staff enhanced the progress of TP students.
Evaluation Design Quasi-Experimental and Non-Experimental: Data were collected from Program Transition Coordinators and other TSPP staff, classroom teachers, students, and parents. TSPP students were surveyed in both the fall and spring of the 2001–2002 school year. For two outcome areas, classroom grades and school attendance, evaluators compared TSPP students (n = 116) to all TP students in the six schools who were not involved in TSPP (n = 1,323). No preprogram characteristics (e.g., 2000–2001 grades/test scores) were available for the two groups, so the evaluators were unable to examine whether the treatment and comparison groups looked similar on such indicators before participating in their respective programs. Due to various data collection complications, the final TSPP sample for students who completed spring surveys totaled 78.
Data Collection Methods Document Review: Program Transition Coordinators at each site filed reports that were analyzed to determine staff perception of progress toward achieving the program's desired outcomes.

Interviews/Focus Groups: Classroom teachers selected by program staff as representative of teacher experiences with the after school program were interviewed to relate their perspective on individual TSPP students. Evaluators interviewed a total of 12 teachers, two at each school.

Secondary Source/Data Review: School attendance data, test scores, and grades earned in math and English Language Arts (ELA) classes were collected from BPS records.

Surveys/Questionnaires: TSSP students completed surveys about: academic work habits (e.g., do you study hard for tests?); relationships with teachers, peers, and parents; school attendance; willingness to try in school; personal, social, and academic satisfaction; opinions on the value of education; enthusiasm for school involvement; pro-social and problem behaviors; parent participation in homework/school related issues; parental emphasis on the importance of school; parental participation in events/conferences; parental communication and relationships with program staff and school faculty; and various other perceptions of the program and school.

Parent surveys asked about: the parents' involvement in the program and in the child's school, perceived benefits of the program for the child, and perceived benefits of the program for the parents. A total of 65 parents were surveyed, and 40 completed an addendum to the parent survey asking about perceptions of how the program helped the parents balance work, education, and child care. Staff surveys asked about perceived benefits of the program for the child, their behavior (social and academic) in the program, and about characteristics of the child. Staff answered surveys for a total of 77 TSPP students.

Test/Assessments: Students' academic benchmark test scores were assessed in English/Language Arts using the Development Reading Assessment (DRA) (third grade only) and the Scholastic Reading Inventory (SRI) (fourth through eighth grade). Mathematics scores were assessed through the Boston Math Tasks and the Mid-Year Math Assessment.
Data Collection Timeframe Data were collected during the 2001–2002 school year.


Findings:
Formative/Process Findings

Parent/Community Involvement Over nine in ten parents (91%) reported that work schedules precluded their regular program participation. Periodic communication between parents and program was still maintained, however, as 80% of parents recalled speaking to the Program Transition Coordinator at least four times during the year.

TSPP parents who reported higher levels of involvement and interaction with their child's school also reported higher levels of positive regard for the program.
Recruitment/Participation Across all program sites, attendance was 75%, meaning that, on average, TSPP children attended for three out of every four hours the program was in operation during the school year. Average attendance rates varied among the sites from a high of 92% to a low of 44%.

TSPP students with parents who were highly involved in the program had higher program attendance. A one-point difference on a five-point measure of staff reported parent involvement (ranging from “not at all” to “a great deal”) was associated with a 2.19-point increase in program attendance, the metric of which was percentage attendance (i.e., total number of days attended divided by total number of days program was in operation).

Prior to their TSSP participation, over 80% of the students reported in the fall 2001 survey that they attended no activity or program after school.

One-fifth of TSPP students were in third grade, 27% were in fourth grade, 14% were in fifth grade, 4% were in sixth grade, 21% were in seventh grade, and 14% were in eighth grade.

Sixty-four percent of TSSP students were male and 36% were female.

Thirty-four percent of TSSP students were Black/African American, 34% were Asian, 19% were Hispanic/Latino(a)/ Spanish, 8% were white, and 5% had missing racial/ethnic status data.

Thirty-one percent of TSSP students were classified as special education students.

Twenty-two percent of TSSP students were bilingual.

Seventy-six percent of TSSP students were eligible for free or reduced-price lunch.


Summative/Outcome Findings

Academic In the spring of 2002, across all sites and grade levels, a minority of participants met the minimum standard for grade-level competency on the standardized test measuring reading ability (44%) and the grade-level proficiency standard (21%). In math, 73% met the minimum competency grade. No math proficiency data were available.

Approximately 40% of TSPP elementary students earned a grade of “meeting or exceeding the standard” in reading and math. (Grades are not given at the elementary level. Teachers report that students are “not meeting,” “minimally meeting,” “meeting,” or “exceeding” the standard.) Among TSPP middle school students, 27% earned a “B” or better in reading and 29% earned a “B” or better in math.

Seventy-seven percent of TSPP students across all grades increased their scores on BPS standardized reading tests from fall to spring.

At all grade levels, the percentages of students who met minimum competency in reading increased from fall to spring. These gains were largest in fourth grade (27% meeting the minimum in the fall, compared to 80% in the spring) and smallest in eighth grade (20% meeting the minimum in the fall, compared to 27% in the spring). In addition, at all grade levels except sixth, the percentages of students who met reading proficiency increased. These gains were largest in fourth grade (0% meeting proficiency in the fall, compared to 47% in the spring) and smallest in seventh grade (0% meeting the minimum in the fall, compared to 4% in the spring).

The rate of TSPP students promoted to the next grade (91%) exceeded the promotion of the TP-only students (84%). Evaluators report that significance levels could not be obtained for promotion data because the data was too positively skewed.

TSPP students in both elementary and middle school did significantly better on their final ELA grade than TP-only students (p<.001 and p<.05, respectively) and both elementary and middle school TSPP students did significantly better on their final math grade than TP-only students (p<.005). For example, among TSPP elementary students, 92% earned passing grades in ELA and 77% passed math class, compared to 54% and 51% of TP-only students. Among TSPP middle school students, 77% earned passing grades in ELA and 69% in math, compared to 60% and 30% of TP-only students.

Students who experienced the greatest gains in reading among TSPP students were those who reported feeling less confident in school. That is, the student measure of success in an academic setting was significantly related to change in SRI scores (t=2.73; p=.001) when controlling for school level (i.e., elementary or middle school level).

TSPP students' average school attendance rate was 7.55% higher than TP-only students' and this difference was statistically significant (p=.001). TSPP students were absent an average of six school days of the 2001–2002 school year. In contrast, TP-only students missed an average of 19 school days. Anecdotal reports suggested that the Program Transition Coordinator was essential to ensuring higher school attendance, giving careful attention to students' needs, thereby imparting an implicit message to students that their presence mattered.

Nearly nine in ten parents of TSPP students believed that the program helped their child learn basic skills in reading, writing, and math, and eight in ten agreed that if their child did not participate in the program, their child's schoolwork would suffer.

Nearly eight in ten TSPP students reported that the program helped them improve their grades. Approximately seven in ten reported that it helped them spend more time doing homework and improve their reading skills. Six in ten reported that the program helped improve their math skills.

Over half of TSPP students reported that they tried harder at school and liked school more because of the program, and over half concurred that the program improved their school attendance.

Staff reported that, for nine in ten students, the program helped to improve their homework and schoolwork habits.
Family Nearly three-fifths of TSPP students agreed that the program helped them get along better with their parents (59%).

Three-quarters of parents of TSPP students (75%) declared that the program helped them connect with their child's teachers and that their involvement in their child's school increased because of their child's involvement in this after school program.

Eighty percent of parents reported that their child's participation in the after school program enabled them to understand their child's schoolwork better.

Over three-fourths of parents (78%) reported that because of their child's participation in the after school program, it was easier to keep their job, and 35% of parents claimed their child's participation made it easier for them to go to school to further their own education.
Youth Development Nearly seven in ten TSPP students reported that the program helped increase their artistic interest/knowledge, and approximately six in ten reported that it helped them acquire related practical/technical skills.

Between 85% and 95% of staff reported that large majorities of students showed organizational skills, assumed responsibility, and worked carefully and neatly.

Roughly two-thirds of TSPP students believed that the program improved their ability to relate to others of a different culture (65%) and to get along with their friends (62%).

Nearly three-fifths of TSPP students agreed that the program helped them get along better with their teachers (59%) and stay out of trouble (57%).

Staff believed that for nearly all TSPP students, participation in TSPP contributed to improvements in their positive behavior (98%) and ability to get along well with other students (95%).

Staff reported that some negative behaviors increased from fall the spring. The following are percentage increases from fall to spring of the percentage of participants who: talked back when corrected (12%), argued with others (18%), fought with others (18%), and liked to be alone (22%).

© 2016 Presidents and Fellows of Harvard College
Published by Harvard Family Research Project