You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.

www.HFRP.org

The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.

Terms of Use ▼

Program Description

Overview CHAMPS is a program designed to add an academic enrichment component to existing after school programs in the School District of Palm Beach County (SDPBC) in Florida. CHAMPS’ objectives are to (a) increase after school program staff’s skills in facilitating academic enrichment curriculum and (b) increase CHAMPS participants’ academic achievement.
Start Date fall 2002
Scope local
Type after school
Location suburban
Setting public schools, community-based organizations
Participants kindergarten and elementary school students (grades K–6)
Number of Sites/Grantees 10 in 2002–2003; 40 in 2003–2004; 38 in 2004–2005
Number Served 892 in 2002–2003; 3,771 in 2003–2004; 3,775 in 2004–2005
Components CHAMPS is designed to be implemented in the same way across all program sites. During the after school programs at the sites, youth rotate through a variety of activities—including snack time, free time to play outdoors, homework assistance, CHAMPS activities, and indoor recreational activities—tailored to meet the interests of youth and resources of each site. Youth participate in CHAMPS for 1 hour per day, 4 days per week. On average, youth complete one CHAMPS lesson per day.

CHAMPS operates under the guidance of CHAMPS staff and a head teacher at each program site. Each school principal and after school director selects a certified school-day teacher to act as the head teacher for the CHAMPS program site for a total of 8 hours per week. This head teacher mentors the instructors and performs the following duties: (a) reviewing the instructors’ lessons, (b) overseeing implementation of the lessons in the classroom, (c) observing and coaching instructors with regard to their teaching skills, (d) preparing materials for the instructors, (e) assisting teachers who need help with facilitation techniques, (f) completing CHAMPS program paperwork, and (g) requesting CHAMPS supplies. CHAMPS instructors complete training in facilitative teaching strategies prior to implementing the program.

CHAMPS activities consist of the implementation of academic curricula in existing after school programs. In the 1st year of implementation, after school program staff facilitated the curriculum Foundations Travels, in the 2nd year, the curriculum Foundations Celebrates, and in the 3rd year, the curriculum Blast Off. The content and activities of all sets of curricula were aligned to the state of Florida’s Language Arts Literacy and Mathematics standards by the curricula developers. Although the curricula were designed for students in third grade and above, instructors reduced the complexity of the curricula so that the content and activities were developmentally appropriate for students in lower grade levels as well.

CHAMPS was implemented in the following three settings:

1. School Age Child Care Programs (SACCs): SACCs are financially self-supporting programs in which participating families pay a fee for each child who attends. To encourage youth from lower income families and underserved groups to participate in the program, CHAMPS provides scholarships that are distributed based on the judgment of the administration at each school. SACC staff must hold high school or general education diplomas, complete first aid and CPR training, and training to care for preschoolers.

2. Beacon Programs: Beacon schools partner with community-based organizations to offer after school childcare and additional programs that create positive changes in youth development, academic achievement, and healthy family functioning. Youth from the community, who do not necessarily attend the Beacon school during the day, are eligible to participate in after school programs offered at Beacon schools. Youth participate in CHAMPS at no cost.

3. Community Based Organizations (CBOs): CBOs are designed to serve the individual needs of the communities in which they are located. Each organization’s mission and operating procedures are unique, although all provide after school programs for school-aged children.
Funding Level $382,600 in 2002–2003; $1.35 million in 2003–2004; $1.35 million in 2004–2005
Funding Sources The Toppel Family Foundation, the Children’s Services Council of Palm Beach County, and the Picower Foundation

Evaluation

Overview The evaluation sought to understand how the addition of an academic enrichment component to after school programs supports improved student achievement and increased professionalism of after school staff.
Evaluators Kimberly A. Knutson, School District of Palm Beach County
Evaluations Profiled Evaluation of the CHAMPS After-School Program: FY2003

CHAMPS SACC Program Evaluation Report

CHAMPS After-School Program Outcome Evaluation Year Two: FY2004
Evaluations Planned CHAMPS’ academic outcomes will be evaluated through the 2004–2005 school year. The final evaluation will address the question of how the effect on student achievement produced by CHAMPS compares to the effect on student achievement produced by (a) the standard academic enrichment activities offered through the district SACC program and (b) standard district after school care with homework help and no academic enrichment activity component.
Report Availability Knutson, K. A. (2003). CHAMPS SACC program evaluation report. Coral Springs, FL: Author.

Knutson, K. A. (2003). Evaluation of the CHAMPS after-school program: FY2003. Coral Springs, FL: Author.

Knutson, K. A. (2004). CHAMPS after-school program outcome evaluation year two: FY2004. Coral Springs, FL: Author.

Knutson, K. A. (2005). CHAMPS after-school program outcome evaluation year three: FY2005. Coral Springs, FL: Author.

Contacts

Evaluation Kimberly A. Knutson, Ed.D.
Specialist, Testing and Evaluation
School District of Palm Beach County
3370 Forest Hill Blvd., Suite 228
West Palm Beach, FL 33406-5871
Tel: 561-434-8782
Fax: 561-963-3842
Email: Knutson@palmbeach.k12.fl.us
Program Judy Nee
Director, CHAMPS Program
8487 155th Place North
Palm Beach Gardens, Florida 33418
Tel: 561-741-4641
Email: judynee@bellsouth.net
Profile Updated January 5, 2006

Evaluation 1: Evaluation of the CHAMPS After-School Program: FY2003


Evaluation Description

Evaluation Purpose To answer three questions: (a) Was the percentage of CHAMPS participants who scored at grade level or above on multiple standardized achievement measures greater than the percentage of comparable nonparticipants, (b) were the average spring standardized achievement test scores of CHAMPS participants greater than those of comparable nonparticipants, and (c) did CHAMPS participants achieve greater reading and math standardized achievement test score gains than comparable nonparticipants?
Evaluation Design Quasi-Experimental: The evaluation compared 478 second through fifth grade CHAMPS participants to 447 similar nonparticipants. The comparison group consisted of a random sample of youth stratified by grade, race/ethnicity, free/reduced-price lunch status (FRL), and limited English proficiency (LEP) status who were selected from a pool of youth not attending CHAMPS from each of the 10 school sites. The comparison group was matched proportionally to the program group on these demographic indicators. Subgroup analyses were done by grade and within grades for race/ethnicity, FRL status, and LEP status.
Data Collection Methods Secondary Source/Data Review: Achievement test score data were obtained from SDPBC.

Test/Assessments: The achievement test scores obtained from SDPBC consisted of the Florida Comprehensive Assessment Test of Sunshine State Standards (FCAT), the Florida Comprehensive Assessment Test-Norm Referenced Test (FCAT-NRT), and the Scholastic Reading Inventory (SRI). The FCAT is a criterion-referenced test that measures the degree to which students have mastered Florida’s standards for reading, writing, and math. The FCAT-NRT measures how well students perform in reading and math compared to a national sample of students. SRI scores measure reading comprehension. SRI scores are available beginning in second grade, while FCAT and FCAT-NRT scores are available beginning in third grade.

Using these test results, four measures were examined: (a) the percentage of youth performing at grade level on the FCAT, FCAT-NRT, and SRI in the spring; (b) average performance on the FCAT Developmental Scale (FCAT-DS), FCAT-NRT Normal Curve Equivalent (NCE), and SRI scores in the spring; (c) the amount of spring to spring gain on the FCAT-DS and FCAT-NRT, and the amount of fall to spring gain on the SRI; and (d) the effect size of statistically significant results for each analysis. (For test score gains, evaluators also calculated “effect size differences,” which are the differences in calculated pretest to posttest effect sizes for the treatment and comparison groups.) Effect size differences were considered meaningful if they reached the following thresholds: .20 = small difference, .50 = moderate difference; and .80 = large difference.
Data Collection Timeframe Data were collected for fiscal years 2002 and 2003.


Findings:
Summative/Outcome Findings

Academic The percentage of CHAMPS youth who scored above grade level did not differ significantly from the percentage of comparison group youth for any of the three tests, overall or for any subgroup.

The average reading SRI scores for CHAMPS second graders did not differ from those in the comparison group, overall or for any subgroup

White CHAMPS third graders had significantly lower average math FCAT and FCAT-NRT scores than their comparison group counterparts (p = .04 and p = .02, effect sizes of -.36 and -.40, respectively). No other significant differences were found for third graders’ math FCAT and FCAT-NRT scores, either overall or for any subgroup.

FRL CHAMPS fourth graders had significantly higher average reading and math FCAT scores than their comparison group counterparts (p = .03 and p = .05, effect sizes of .50 and .47, respectively). No other significant differences were found for fourth graders’ FCAT scores, either overall or for any subgroup.

CHAMPS fifth graders had significantly higher average SRI reading scores than their comparison group counterparts (p < .04, effect size of .27). This was also true for African American CHAMPS fifth graders (p < .01, effect size of .59). No other significant differences were found for fifth graders’ SRI scores, either overall or for any subgroup.

LEP CHAMPS third graders demonstrated smaller gains on the SRI than their comparison group counterparts (effect size difference = -.24). No other meaningful effect size differences were found for third graders’ SRI scores, either overall or for any subgroup.

CHAMPS fourth graders experienced larger FCAT reading score gains than their comparison group counterparts (effect size difference = .27). This was also true for CHAMPS fourth graders who were African American, FRL status, and LEP status (effect size differences = .40, .38, and .67, respectively).

CHAMPS fourth graders experienced smaller FCAT-NRT reading gains than their counterparts in the comparison group (effect size difference = -.21). African American CHAMPS fourth graders experienced smaller gains than their comparison group counterparts (effect size difference = -.55). Also, FRL CHAMPS fourth graders experienced a slight decline while their counterparts in the comparison group experienced gains (effect size difference = -.65).

On the SRI, no overall differences were found between the CHAMPS and comparison group fourth graders, but FRL and LEP CHAMPS fourth graders experienced greater gains than did their counterparts in the comparison group (effect size differences = .41 and .34, respectively).

No overall differences were found between CHAMPS and comparison group fourth graders for math scores on either the FCAT or the FCAT-NRT. However, CHAMPS fourth graders who were African American, FRL status, and LEP status all experienced greater FCAT gains than their comparison group counterparts (effect size differences = .41, .58, and .39, respectively), while African American CHAMPS youth and FRL CHAMPS fourth graders experienced smaller FCAT-NRT gains than their comparison group counterparts (effect size differences = -.30 each).

CHAMPS fifth graders experienced smaller FCAT reading score gains than their comparison group counterparts (effect size difference = -.23), while White CHAMPS fifth graders experienced smaller gains than their comparison group counterparts (effect size difference = -.27).

For fifth graders’ FCAT-NRT reading scores, no differences were found either overall or for any subgroup.

CHAMPS fifth graders demonstrated greater SRI score gains than comparison group youth (effect size difference = .27). This was also true for African American, White, and FRL subgroups (effect size differences = .43, .20, and .21, respectively).

CHAMPS fifth graders demonstrated greater gains in FCAT math scores than comparison group youth (effect size differences = .60). This was also true for the African American and FRL subgroups (effect size differences = 1.53 and .76, respectively). On the FCAT-NRT, no differences were found for fifth graders, either overall or for any subgroup.

Evaluation 2: CHAMPS After-School Program Evaluation Report


Evaluation Description

Evaluation Purpose To answer two questions: (a) What project site implementation activities are helping CHAMPS instructors to acquire and improve their skills at facilitating the academic enrichment curriculum, and (b) what activities are helping CHAMPS instructors to support improved academic achievement for youth participating in SACC programs?
Evaluation Design Non-Experimental: Data were collected from staff and youth at all nine sites through interviews, focus groups, and surveys.
Data Collection Methods Interviews/Focus Groups: Phone interviews were conducted with all nine SACC program directors about CHAMPS’ impact on the SACC program and their professional development.

Focus groups, conducted with 39 CHAMPS instructors across the nine sites, dealt with staff’s role changes since becoming CHAMPS instructors and what activities were helpful as they developed as CHAMPS instructors.

Surveys/Questionnaires: All nine head teachers and 41 out of 52 CHAMPS instructors at the nine CHAMPS schools completed surveys. The survey asked questions concerning CHAMPS instructors’ professional development activities, the skills needed to be a CHAMPS instructor, CHAMPS instructors’ levels of responsibility and professionalism, program implementation, feelings about the program’s success, and student achievement.

All 892 youth participants were asked to complete a survey. A total of 440 surveys were returned from CHAMPS participants across the nine sites. The youth survey focused on demographic background characteristics, feelings about CHAMPS, and perceptions of CHAMPS’ impact.
Data Collection Timeframe Interviews were conducted November through December 2002. Surveys were collected in January 2003. Focus groups were conducted in March 2003.


Findings:
Formative/Process Findings

Staffing/Training Nine of the 10 schools hired a certified teacher who taught at an SDPBC elementary school to be the lead teacher for the program. The 10th hired the CHAMPS curriculum coordinator, who was a certified teacher but not an SDPBC teacher, as the lead teacher.

The curriculum coordinator evaluated each instructor’s performance on five skills during the first half of the year: planning, instruction, classroom environment, professionalism, and overall experience for youth. Of the CHAMPS instructors evaluated, 64% were rated above average, 23% average, and 13% below average. Instructors performing below average received corrective feedback that included being closely monitored by the head teacher and/or being paired with another instructor.

Eighty-seven percent of CHAMPS instructors and 89% of head teachers thought the in-service CHAMPS training prior to implementation was helpful for preparing SACC staff to implement the academic enrichment curriculum.

Once instructors began implementing CHAMPS, 83% found one-on-one coaching with the head teacher to be very helpful, 85% reported that the evaluation of their performance with the CHAMPS curriculum coordinator gave them feedback on their teaching skills, and 79% reported that this evaluation was helpful.

The majority of CHAMPS instructors reported that the head teacher supported them in the following ways: (a) providing materials on a daily basis (98%), (b) modeling activities as they needed help (93%), (c) observing their classroom activities on a regular basis (93%), (d) providing regular feedback on their skill implementing the curriculum (90%), and (e) providing information on each topic covered in the curriculum (85%).

The majority of head teachers and instructors reported discussing the following issues at their school on a daily basis: (a) helping children communicate effectively with each other (78%), (b) classroom management to help students behave positively (69%), (c) planning and implementing educational activities (65%), (d) planning social skill building exercises (63%), and (e) planning instruction to meet multiple learning styles of students (61%).

Eighty-three percent of CHAMPS instructors reported that working in CHAMPS improved their professional development and improved their confidence, and 80% strongly agreed that their work in CHAMPS was a positive experience.

CHAMPS instructors agreed that the skill needed to be a CHAMPS instructor was equal to the skill needed to be a teacher (85%) and exceeded the skill needed to be a teacher’s aide (65%) or childcare worker (75%). Head teachers agreed that the skill needed to be a CHAMPS instructor was equal to the skill needed to be a teacher (56%) or a teacher’s aide (100%) and was equal to or above the skill needed to be a childcare worker (100%).

CHAMPS instructors reported that the responsibility of CHAMPS instructors was equal to or higher than the responsibility of a teacher (92%) and exceeded the responsibility of a teacher’s aide (70%) or childcare worker (68%). Head teachers reported that the responsibility of a CHAMPS instructor was equal to a teacher (67%) and a teacher’s aide (100%) and exceeded the responsibility of a childcare worker (68%).

CHAMPS instructors reported that the professionalism of CHAMPS instructors was equal to the professionalism of a teacher (85%), equal to or above the professionalism of a teacher’s aide (98%), and exceeded the professionalism of a childcare worker (60%). Head teachers reported that the professionalism of a CHAMPS instructor was equal to a teacher (85%) and a teacher’s aide (99%) and was above the professionalism of a childcare worker (60%).


Summative/Outcome Findings

Academic The majority of CHAMPS head teachers and instructors agreed that CHAMPS youth showed moderate to great improvement in critical thinking (92%), listening (92%), and visual presentation skills (90%).

CHAMPS head teachers and instructors reported seeing improvement of youth’s verbal communication (90%) and writing skills (84%).

CHAMPS head teachers and instructors reported that CHAMPS improved youth’s reading performance (76%).

CHAMPS head teachers and instructors reported that CHAMPS had no effect to moderate improvement on youth’s math performance (90%).

When asked how they felt while at CHAMPS, the majority of youth participants reported being good “always” or “sometimes” at math (91%), thinking (90%), listening (90%), reading (88%), art (86%), writing (83%), and science (78%).

Youth reported that when they came to CHAMPS they always or sometimes enjoyed learning (87%) and coming to school (83%).
Youth Development CHAMPS head teachers and instructors reported that CHAMPS either greatly or moderately improved youth’s ability to work cooperatively with other youth (92%), attitude toward learning (90%), attention and behavior (88%), attitude toward school (86%), self-confidence in learning (84%), self-esteem (84%), time on task (83%), and interest in school (78%).
Youth reported that because of the work they did in CHAMPS, they always knew they could learn new things (77%), were always proud of their work (74%), and always felt smart (64%).

A slight majority of youth (53%) reported that they always paid attention and followed rules and that they always worked well with other youth (66%) when they were at CHAMPS. Additionally, 60% of youth reported that since coming to CHAMPS, they always liked to work with other students in their regular school-day classes.

Evaluation 3: CHAMPS After-School Program Outcome Evaluation Year Two: FY2004


Evaluation Description

Evaluation Purpose To answer the following questions: (a) Were average spring achievement test scores of CHAMPS participants greater than those of comparable nonparticipants, (b) did CHAMPS participants achieve greater reading and math gains on achievement test scores than comparable nonparticipants, (c) was there a difference in the reading and math achievement by the types of sites in which CHAMPS was implemented, and (d) was the reading and math achievement of CHAMPS participants at the SACC sites different in the 2nd year of implementation than in the 1st year?
Evaluation Design Quasi-Experimental: Of the 3,771 youth from pre-kindergarten through sixth grade enrolled in CHAMPS, 2,824 attended at least 80% of the total number of program days that CHAMPS was conducted during the 2003–2004 school year. Of these 2,824 participants, only second through fifth grade participants enrolled in school before October 1, 2003, and who attended the same school for the entire school year were included in the evaluation, for a total of 1,830 youth.

Participants were compared to a sample of 1,761 youth not attending CHAMPS who matched the demographic characteristics of CHAMPS participants (grade, race/ethnicity, FRL status, and LEP status). In addition to being comparable demographically, comparison group youth attended the same schools during the day as CHAMPS participants. All youth in the comparison group had the opportunity to sign up for after school care but did not. Treatment and comparison group youth had similar 2002–2003 achievement test scores, with the only significant difference being among fourth graders at Beacon sites, where CHAMPS participants were less likely to have low reading achievement scores and less likely to have low or high math achievement scores than comparison group youth (p < .05). Subgroup analyses were done by grade and by site and within grades and sites for race/ethnicity, FRL status, and LEP status.
Data Collection Methods Secondary Source/Data Review: Achievement test scores and demographic data were obtained from SDPBC.

Test/Assessments: The achievement test scores obtained from SDPBC consisted of the Florida Comprehensive Assessment Test of Sunshine State Standards (FCAT) and the Scholastic Reading Inventory (SRI). SRI scores are first available beginning in second grade, while FCAT scores are first available beginning in third grade. The FCAT is a criterion-referenced test that measures the degree to which students have mastered the Sunshine State Standards for reading, writing, and math. SRI scores measure reading comprehension.

Using these test results, three measures were examined: (a) the average performance on the spring FCAT Developmental Scale (FCAT-DS) and SRI scores, and (b) the amount of spring to spring gain on the FCAT-DS and the amount of fall to spring gain on the SRI, and (c) the effect size of statistically significant results of each analysis. (For test score gains, evaluators also calculated “effect size differences,” which are the differences in calculated pretest to posttest effect sizes for the treatment and comparison groups). Effect size differences were considered meaningful if they reached the following thresholds: .20 = small difference, .50 = moderate difference, and .80 = large difference.
Data Collection Timeframe Data were collected for fiscal year 2004.


Findings:
Summative/Outcome Findings

Academic Among second graders at Beacon sites, the average SRI scores for all, African American, FRL, and LEP CHAMPS youth were significantly higher than their counterparts in the comparison group (p < .05). The effect sizes were small for all youth and the FRL subgroup, and moderate for African American and LEP youth.

Among second graders at SACC sites, there were no significant differences on SRI scores overall, although the average SRI scores for CHAMPS African American, Hispanic, and FRL youth were significantly higher than youth in respective comparison groups (p < .05). The effect sizes were small for African American and FRL youth and moderate for Hispanic youth.

Among second graders at CBO sites, the average SRI scores for CHAMPS youth did not differ from comparison group youth, either overall or by subgroup.

Among third graders at SACC sites, there were no significant differences on the SRI or FCAT overall, although the average SRI scores for CHAMPS African American, LEP, and FRL youth were significantly higher than youth in their respective comparison groups (p < .05). The effect sizes were small for FRL youth and moderate for African American and LEP youth. In addition, the average FCAT math score for African American CHAMPS youth was significantly higher than for comparison group youth (p < .05), although the effect size was small.

Among third and fourth graders at CBO and Beacon sites as well as fifth graders at SACC sites, the average FCAT and SRI reading and math scores for CHAMPS youth did not differ from their comparison group counterparts, overall or by subgroup.

Among fourth graders at SACC sites, there were no significant differences on the SRI or FCAT overall, although Hispanic CHAMPS youth’s average SRI-I score was significantly higher than the Hispanic youth in the comparison group (p < .05).
Among fifth graders at Beacon sites, there were no significant differences on the SRI or FCAT overall, although the average FCAT math and reading scores of African American CHAMPS youth were significantly higher than their comparison group counterparts (p < .05). The effect size was moderate for both reading and math.

Among third graders at Beacon sites, all, African American, and FRL CHAMPS youth demonstrated less gain on the FCAT reading than their counterparts in the comparison group (p < .05). The effect sizes were small.

Among third graders at CBO sites, CHAMPS youth demonstrated less gain on the FCAT reading and math than comparison group youth. The effect size was moderate for both. In math, CHAMPS African American and FRL youth demonstrated significantly less gain than youth in their respective comparison groups (p < .05). The effect sizes were moderate for both groups. On the SRI-I, the gains demonstrated by all youth, African American youth, LEP youth, and FRL youth participating in CHAMPS was significantly greater than the amount of gain demonstrated by youth in the comparison groups (p < .05). The amount of gain was small for all youth, African American youth, and FRL youth and moderate for LEP youth.

Among third graders at SACC sites on the SRI-I, there were no significant differences on the SRI or FCAT overall, although White CHAMPS youth demonstrated less gain than White youth in the comparison group (p < .05). The effect size was small. LEP CHAMPS youth demonstrated greater gains than LEP comparison group youth (p < .05). The effect size was moderate. On FCAT reading, African American and FRL youth demonstrated greater gains than their comparison group counterparts (p < .05). The effect sizes were moderate. On FCAT math, African American and FRL youth demonstrated greater gains than their comparison group counterparts (p < .05). The effect size was moderate for African American and small for FRL youth.

Among fourth graders at Beacon sites, there were no significant differences on the SRI or FCAT overall, although Hispanic and LEP CHAMPS youth demonstrated less gain on FCAT reading than their comparison group counterparts (p < .05). The effect size was large. In math, all, Hispanic, and LEP CHAMPS youth demonstrated greater gains than their comparison group counterparts (p < .05). The effect size was small for all youth, large for Hispanic youth, and moderate for LEP youth.

Among fourth graders at CBO sites, there were no significant differences on the SRI or FCAT overall, although LEP CHAMPS youth demonstrated greater gains on the SRI-I than comparison group youth (p < .05). The effect size was large. On the FCAT math, all, LEP, and FRL CHAMPS youth demonstrated greater gains than their counterparts in the comparison group (p < .05). The effect sizes were small.

Among fourth graders at SACC sites, there were no significant differences on the FCAT or SRI overall, although all and FRL CHAMPS youth demonstrated greater SRI-I gains than their comparison group counterparts (p < .05). The effect size was small for all youth and moderate for FRL youth. On FCAT reading, all, African American, Hispanic, and FRL CHAMPS youth demonstrated less gain than their comparison group counterparts (p < .05). The effect size was small for all groups. On the FCAT math, all, African American, and Hispanic CHAMPS youth demonstrated greater gains than their comparison group counterparts (p < .05). LEP CHAMPS youth demonstrated less gain than their comparison group counterparts (p < .05). The effect size for all groups was small.

Among fifth graders at Beacon sites, there were no significant differences on the FCAT or SRI overall, although all CHAMPS youth and African American CHAMPS youth demonstrated greater SRI-I gains than comparison group youth (p < .05). The effect sizes were small. On the FCAT math, all, African American, Hispanic, LEP, and FRL CHAMPS participants demonstrated greater gains than comparison group youth. The effect sizes were small. White CHAMPS youth demonstrated less gain on the FCAT math than their comparison group counterparts. The effect size was large.

Among fifth graders at CBO sites, there were no significant differences on the SRI or FCAT overall, although LEP youth demonstrated greater SRI-I gains than their comparison group counterparts (p < .05). The effect size was small. On the FCAT math, all, African American, and FRL CHAMPS youth demonstrated greater gains than their comparison group counterparts (p < .05). The effect size was small. LEP CHAMPS youth demonstrated less gain on the FCAT math than their comparison group counterparts (p < .05). The effect size was moderate.

Among fifth graders at SACC sites, there were no significant differences on the SRI or FCAT reading, although all, African American, White, LEP, and FRL CHAMPS participants demonstrated greater gains on the FCAT math than comparison group youth (p < .05). The effect size for all groups was small.

When comparing estimated effects in reading across program site contexts, the following findings emerged: (a) CHAMPS third graders at CBO program sites performed better relative to their comparison group counterparts on SRI-I than did CHAMPS third graders at SACC and Beacon sites, (b) CHAMPS third graders at SACC sites performed better on FCAT reading relative to their comparison group counterparts than did CHAMPS third graders at Beacon and CBO sites, (c) CHAMPS fourth graders at SACC sites performed better on SRI-I relative to their comparison group counterparts than did CHAMPS fourth graders at Beacon and CBO sites, (d) LEP CHAMPS fourth graders at CBO sites performed better on SRI-I relative to their comparison group counterparts than did LEP CHAMPS fourth graders at Beacon and SACC sites, and (e) Hispanic CHAMPS fourth graders at Beacon sites performed better on FCAT reading relative to their comparison group counterparts than did Hispanic CHAMPS fourth graders at SACC or CBO sites.

When comparing estimated effects on FCAT math scores across program site contexts, the following findings emerged: (a) CHAMPS third graders at SACC sites performed better relative to their comparison group counterparts than did CHAMPS third graders at Beacon or CBO sites, (b) African American CHAMPS fourth graders at SACC sites performed better relative to their comparison group counterparts than did African American CHAMPS fourth graders at Beacon and CBO sites, (c) Hispanic and LEP CHAMPS fourth graders at Beacon sites performed better relative to their comparison group counterparts than did Hispanic and LEP CHAMPS fourth graders at SACC and CBO sites, (d) FRL CHAMPS fourth graders at CBO sites performed better relative to their comparison group counterparts than did FRL CHAMPS fourth graders at SACC and Beacon sites, and (e) CHAMPS fifth graders at Beacon sites performed better relative to their comparison group counterparts than did CHAMPS fifth graders at SACC and Beacon sites.

When comparing effect size estimates between year 1 and year 2 of the evaluation for reading, the following findings emerged: (a) The estimated impact of CHAMPS on the reading performance of LEP third graders was greater in year 2 than in year 1; (b) on SRI-I, the estimated impact of CHAMPS on the reading performance of fourth graders was greater in year 2 than in year 1 for all and FRL youth; (c) on FCAT, the estimated impact of CHAMPS on the reading performance of fourth graders was less in year 2 than in year 1; and (d) on SRI-I, the estimated impact of CHAMPS on the reading performance of fifth graders was less in year 2 than in year 1.

When comparing effect size estimates between year 1 and year 2 of the evaluation for math, the following findings emerged: (a) The estimated impact of CHAMPS on the math performance of all and Hispanic fourth graders was greater in year 2 than in year 1; (b) the impact of CHAMPS on the math performance of FRL and LEP fourth graders was less in year 2 than in year 1; (c) the estimated impact of CHAMPS on the math performance of all, White, and LEP fifth graders was greater in year 2 than in year 1; and (d) the estimated impact of CHAMPS on the math performance of African American and FRL fifth graders was less in year 2 than in year 1.

© 2016 Presidents and Fellows of Harvard College
Published by Harvard Family Research Project