You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.

www.HFRP.org

The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.

Terms of Use ▼

Program Description

Overview Sacramento START (Students Today Achieving Results for Tomorrow) is an after school program that provides children from low-income neighborhoods in Sacramento, California, with homework assistance and enrichment activities. The program is a partnership between the city of Sacramento, the county of Sacramento, and six local school districts. START is built around the idea that high quality after school programs promote youth development and build on student assets by helping children form new friendships, develop new interests, and build their self-esteem. This positive youth development, in turn, promotes social and emotional growth and greater attachment to school. START aims to be particularly beneficial to students who are struggling academically.
Start Date 1995
Scope local
Type after school
Location urban
Setting public schools
Participants kindergarten and elementary school students (K–6)
Number of Sites/Grantees 18 elementary schools in 1995–1996; 29 elementary schools in 1998–1999; 40 elementary schools in 1999–2000; 32 elementary schools in 2000–2001; and 37 elementary schools in 2001–2002, and 2002–2003. START involves six school districts in the Sacramento region.
Number Served 4,209 in 1999–2000; 3,820 in 2000–2001; 5,136 in 2001–2002; and 6,939 in 2002–2003
Components Each START program includes three components: homework and tutoring assistance, literacy, and enrichment. The following activities are incorporated in all START programs at least twice a week: (1) reading—group and individual reading opportunities that emphasize accuracy and fluency; (2) literacy and math—activities presented at each grade level to support and reinforce the chosen curriculum at the school site; and (3) enrichment—activities such as art, cooking, crafts, science, health and physical education, cooperative games, sports, drama and dance, music and movement, cultural and holiday-themed activities, field trips, nutrition, gardening, and environmental education.

Each START site is also provided with an array of educational resources, such as games, language review resources, literacy center resources, and activity resources.

Program Leaders serve classes of approximately 20 students. Site Directors supervise Program Leaders and manage programs. Regional Directors supervise five to twelve Site Directors and coordinate activities with school districts and citywide functions (e.g., training, special events, and enrichment activities). A classroom teacher at each school serves as an Academic Alignment Coach to link the START program with the school curriculum and school staff. The Academic Alignment Coach is paid an annual stipend. Recreation Managers and Program Managers supervise the program for the city of Sacramento.
Funding Level $4.8 million (2001–2002)
Funding Sources the city of Sacramento, six local school districts, and numerous corporations, foundations, and individuals
Other The city of Sacramento Department of Neighborhood Services serves as the fiscal agent for the program, employs and trains staff, and prepares reports for funders. START also requires five core training classes for all program staff offering direct services, which cover an array of topics related to providing after school services.


Evaluation

Overview START is evaluated annually to look at program implementation and its impacts on participating youth.
Evaluator Minicucci Associates
Evaluations Profiled Achieving Results: Evaluation Report 1999/2000

Supporting Student Achievement: Evaluation Report 2000/2001

Evaluation Report 2001/2002
Evaluations Planned The 2002–2003 evaluation is in process. Annual evaluations are required by the California Department of Education, which provides major funding for the program.
Report Availability Lamare, J. (1998). Sacramento START: An evaluation report, September 1996 – May 1997. Sacramento, CA: Sacramento Neighborhood Planning and Development Services Department. Summary available at www.aypf.org/rmaa/pdfs/sacSTART.pdf (Acrobat file).

Minicucci Associates. (2001). Achieving results: Evaluation report 1999/2000. Sacramento, CA: Author. Available at www.sacstart.org/pdf/1999-2000_eval.pdf (Acrobat file).

Minicucci Associates. (2001). Supporting student achievement: Evaluation report 2000/2001. Sacramento, CA: Author. Available at www.sacstart.org/pdf/2000-2001_eval.pdf (Acrobat file).

Minicucci Associates. (2002). Evaluation report 2001/2002. Sacramento, CA: Author. Available at www.sacstart.org/pdf/2001-2002_eval.pdf (Acrobat file).


Contacts

Evaluation Cathy Adams Minicucci
Minicucci Associates
1540 River Park Dr., Ste. 212
Sacramento, CA 95815
Tel: 916-920-7800 x224
Fax: 916-649-3161
Email: minicucci@pacbell.net
Program Sacramento START
8795 Folsom Boulevard, Suite 101
Sacramento, CA 95826
Tel: 916-808-6089
Fax: 916-808-1214
Profile Updated May 11, 2004

Evaluation 3: Evaluation Report 2001/2002


Evaluation Description

Evaluation Purpose To examine (1) program implementation strengths, challenges, and emerging solutions and (2) program participants' outcomes in START attendance, school attendance, academic achievement, and behavior and attitudes.
Evaluation Design Quasi-Experimental and Non-Experimental: Eleven sites were chosen for an in-depth examination of implementation issues. To assess the impacts of the program on participating youth, evaluators constructed an evaluation group and a comparison group. The evaluation group consisted of 1,440 third through sixth grade START students who attended START at least 60 days during 2001–2002 and had complete standardized test information for 2000–2001 and 2001–2002. Based on school district records, a total of 814 comparison students were successfully matched to the evaluation group on five characteristics: school, language fluency, gender, grade, and equivalent SAT-9 reading scores for 2000–2001 (within plus or minus 5 Normal Curve Equivalence [NCE] points). The evaluation sample was 33% African American, 26% Hispanic/Latino, 25% Asian/Pacific Islander, 14% white, and 2% other. The comparison group was 23% African American, 28% Hispanic/Latino, 25% Asian/Pacific Islander, 21% white, and 3% other.
Data Collection Methods Interviews/Focus Groups: Individual interviews were conducted during site visits in spring 2002 at 11 schools with 20 Program Leaders, 11 Site Directors, 11 Principals, and 8 Literacy Coaches. In addition, nine focus groups were conducted with groups of 4–8 classroom teachers, and another 18 focus groups were conducted with groups of 8–10 students. Interviews and focus groups concentrated on program implementation.

Observation: Observations were conducted in 25 START classrooms during site visits at 11 schools in spring 2002 to examine START implementation.

Secondary Source/Data Review: School attendance and test score data were collected from data files maintained by school districts and school sites. START attendance data were provided by individual program sites.

Surveys/Questionnaires: Pretest and posttest questionnaires were collected from 240 START students that asked questions about how often in the past 30 days the students wanted to go to school, studied for a test, and felt unsafe at school. Pretest/posttest data were collected from as many START students as possible at the beginning and end of the 2001–2002 year.

Tests/Assessments: Students' SAT-9 test scores were provided by the school districts for both the 2000–2001 and 2001–2002 school years.
Data Collection Timeframe Data were collected during the 2001–2002 school year, including academic records from the current and previous school year.


Findings:
Formative/Process Findings

Activity Implementation School sites varied in the extent of enrichment activities offered. Four of the 11 sites that were visited offered enrichment at least three days per week, while the remaining seven delivered enrichment one day per week. Most stakeholders concurred that more enrichment was necessary, such as art, science, and sports, which would otherwise not be available to many START students. Some sites responded by bringing in visitors and guest speakers to teach enrichment activities such as drama, dance, or art.
Parent/Community Involvement Strengthening ties with parents was found to be a promising practice, including follow up with parents when a child was absent and daily contact with parents when they picked up their children from the program.
Program-School Linkages Based on interviews, focus groups, and observations, the evaluators found that the linkages between START and the regular school day were enhanced when Program Leaders worked directly with classroom teachers, supported by the Literacy Coach and Site Director. Examples of promising practices included: Program Leaders observing classroom teaching, classroom teachers present during START time, and classroom teachers regularly reporting progress to the program. In addition, Literacy Coaches facilitated the schools' communication with both Program Leaders and Site Directors and provided SAT-9 test material to support START in connecting to school curriculum. Other practices contributing to strong program-school linkages were hiring school staff as Program Leaders and inviting START staff to school staff meetings and retreats.

One program challenge identified by the evaluators was the shared use of space with classroom teachers. In particular, some school staff reported in interviews that START left their classrooms untidy or with missing or broken items. START staff alleviated this problem by replacing missing or broken items and voicing teacher concerns to Program Leaders and Site Directors.
Recruitment/Participation START students were approximately evenly divided between males and females, although girls attended START at a slightly higher rate than boys.

The presence of male Program Leaders and the availability of structured competitive sports were found to help in attracting and retaining boys in the upper grades.

Most START students were in grades one through five; participation tapered off slightly at sixth grade. Similarly, attendance days tended to decline somewhat in the upper grades, particularly at sixth grade. Kindergarteners were also eligible to attend, but because many kindergarten students attended school for half a day, kindergarten students made up only 6% of all START students.

START students participated in the program for an average of 107 days during 2001–2002 (out of an average program operation length of 148 days).
Staffing/Training Based on interviews and observations, the evaluators found that the Literacy Coach position played a pivotal role in creating a high quality after school program. Sites varied in their access to a high quality Literacy Coach. Quality Literacy Coaches made daily contact with Site Directors and Program Leaders, provided training on school curriculum, provided ongoing consultation on teaching methodology and classroom management, and served as a liaison between classroom teachers and START staff.

The need to upgrade the quality of Start's instruction and classroom management was a challenge noted by the majority of teachers and principals interviewed and by evaluators during site observations. Practices that were found to help with this problem were strengthening the presence of the Literacy Coach in START classrooms and classroom teachers' mentoring of Program Leaders.

One promising practice (i.e., a practice found to promote a high quality after school program) revealed through interviews with Site Directors was hiring school staff and college students as Program Leaders for START. College students were found to contribute skills in delivering academic instruction and support.

Using the Literacy Coach as a “floating tutor” in START classrooms was found to be a promising practice, providing students with extra academic support from a classroom teacher and also letting Program Leaders observe effective academic instruction and classroom management.

Some sites surveyed students for their preferences for enrichments activities and took these preferences into consideration in their selection in hiring Program Leaders.


Summative/Outcome Findings

Academic Among students in the lowest quartile of reading SAT-9 scores during 2000–2001, the START evaluation group experienced greater growth in reading achievement than comparison group students (12.5 NCE points vs. 3.9 NCE points). Similar patterns prevailed for math SAT-9 scores (19.1 NCE points vs. 8.5 NCE points).

The results for higher performing START students were mixed for reading scores. For the two middle quartiles of achievers in 2000–2001, the second lowest quartile in both groups made slight gains, while the third quartile dropped approximately 10 NCE points. Among the top performing quartile in 2000–2001, the START group's scores dropped approximately 17.5 NCE points, while the comparison group dropped by 4.8 NCE points.

Results for higher performing START students in math were also mixed. Among the second lowest quartile, START students made larger gains than comparison group students (6.7 NCE points vs. 4 NCE points); among the third quartile, both groups dropped slightly (a 3-NCE-point decrease vs. a 0.9-NCE-point decrease); and among the highest quartile, the START group dropped 16.6 NCE points while the comparison group dropped 4.6 NCE points.

The gains in student achievement among low-performing students generally increased with the number of days they participated in START. Low-performing START students who participated between 60 and 90 days gained 10 NCE points in reading and 12 NCE points in math. For 91–120 days of participation, gains were 14 NCE points in reading and 23 NCE points in math. For 121–150 days, gains were 12.5 NCE points in reading and 23 NCE points in math. And for over 150 days, gains were 26 NCE points in reading and 24 NCE points in math.

START students were classified as having good school attendance and had no change in attendance from 2000–2001 to 2001–2002, with six days absent on average for each year.

For the over 200 students in the START evaluation group whose school attendance was classified as a problem (missing over nine days in 2000–2001), attendance improved substantially in 2001–2002; their days absent dropped from an average of 15 to 11 days.

Among the 183 START evaluation group students who participated in both 2000–2001 and 2001–2002, these two-year participants showed a statistically significant growth in math NCE scores (p < .05) and a nonsignificant decline in reading NCE scores.

The average days absent from school for two-year attendees increased slightly from 2000–2001 to 2001–2002 (from 3 days to 4.5 days), but overall these students were classified as good attendees. Among two-year attendees classified as problem attenders (missing more than nine days of school in 2000–2001), the average number of days absent dropped from 13.7 to 7.3.

Among the 60 START evaluation group students who participated in 1999–2000, 2000–2001, and 2001–2002, these three-year participants showed sustained growth in both reading and math NCE scores (47 NCE points in spring of 1999 to 52 NCE points in spring of 2002 in reading; 32 NCE points in spring of 1999 to 43 NCE points in spring of 2002 in math).

School attendance data for three-year attendees showed that this group had consistently good attendance, missing roughly four days of school per year.

The percentage of surveyed students responding that they wanted to go to school everyday in the past 30 days increased from 56% at pretest to 60% at posttest.

Fewer surveyed students reported studying for a test daily at the posttest than on the pretest, but more students reported studying for tests a few times per week at the time of the posttest than at the time of the pretest.
Youth Development The percentage of surveyed students reporting that they never felt unsafe at school increased from 45% at pretest to 71% at posttest.

© 2016 Presidents and Fellows of Harvard College
Published by Harvard Family Research Project