You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.

www.HFRP.org

The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.

Terms of Use ▼

Program Description

Overview CHAMPS is a program designed to add an academic enrichment component to existing after school programs in the School District of Palm Beach County (SDPBC) in Florida. CHAMPS’ objectives are to (a) increase after school program staff’s skills in facilitating academic enrichment curriculum and (b) increase CHAMPS participants’ academic achievement.
Start Date fall 2002
Scope local
Type after school
Location suburban
Setting public schools, community-based organizations
Participants kindergarten and elementary school students (grades K–6)
Number of Sites/Grantees 10 in 2002–2003; 40 in 2003–2004; 38 in 2004–2005
Number Served 892 in 2002–2003; 3,771 in 2003–2004; 3,775 in 2004–2005
Components CHAMPS is designed to be implemented in the same way across all program sites. During the after school programs at the sites, youth rotate through a variety of activities—including snack time, free time to play outdoors, homework assistance, CHAMPS activities, and indoor recreational activities—tailored to meet the interests of youth and resources of each site. Youth participate in CHAMPS for 1 hour per day, 4 days per week. On average, youth complete one CHAMPS lesson per day.

CHAMPS operates under the guidance of CHAMPS staff and a head teacher at each program site. Each school principal and after school director selects a certified school-day teacher to act as the head teacher for the CHAMPS program site for a total of 8 hours per week. This head teacher mentors the instructors and performs the following duties: (a) reviewing the instructors’ lessons, (b) overseeing implementation of the lessons in the classroom, (c) observing and coaching instructors with regard to their teaching skills, (d) preparing materials for the instructors, (e) assisting teachers who need help with facilitation techniques, (f) completing CHAMPS program paperwork, and (g) requesting CHAMPS supplies. CHAMPS instructors complete training in facilitative teaching strategies prior to implementing the program.

CHAMPS activities consist of the implementation of academic curricula in existing after school programs. In the 1st year of implementation, after school program staff facilitated the curriculum Foundations Travels, in the 2nd year, the curriculum Foundations Celebrates, and in the 3rd year, the curriculum Blast Off. The content and activities of all sets of curricula were aligned to the state of Florida’s Language Arts Literacy and Mathematics standards by the curricula developers. Although the curricula were designed for students in third grade and above, instructors reduced the complexity of the curricula so that the content and activities were developmentally appropriate for students in lower grade levels as well.

CHAMPS was implemented in the following three settings:

1. School Age Child Care Programs (SACCs): SACCs are financially self-supporting programs in which participating families pay a fee for each child who attends. To encourage youth from lower income families and underserved groups to participate in the program, CHAMPS provides scholarships that are distributed based on the judgment of the administration at each school. SACC staff must hold high school or general education diplomas, complete first aid and CPR training, and training to care for preschoolers.

2. Beacon Programs: Beacon schools partner with community-based organizations to offer after school childcare and additional programs that create positive changes in youth development, academic achievement, and healthy family functioning. Youth from the community, who do not necessarily attend the Beacon school during the day, are eligible to participate in after school programs offered at Beacon schools. Youth participate in CHAMPS at no cost.

3. Community Based Organizations (CBOs): CBOs are designed to serve the individual needs of the communities in which they are located. Each organization’s mission and operating procedures are unique, although all provide after school programs for school-aged children.
Funding Level $382,600 in 2002–2003; $1.35 million in 2003–2004; $1.35 million in 2004–2005
Funding Sources The Toppel Family Foundation, the Children’s Services Council of Palm Beach County, and the Picower Foundation

Evaluation

Overview The evaluation sought to understand how the addition of an academic enrichment component to after school programs supports improved student achievement and increased professionalism of after school staff.
Evaluators Kimberly A. Knutson, School District of Palm Beach County
Evaluations Profiled Evaluation of the CHAMPS After-School Program: FY2003

CHAMPS SACC Program Evaluation Report

CHAMPS After-School Program Outcome Evaluation Year Two: FY2004
Evaluations Planned CHAMPS’ academic outcomes will be evaluated through the 2004–2005 school year. The final evaluation will address the question of how the effect on student achievement produced by CHAMPS compares to the effect on student achievement produced by (a) the standard academic enrichment activities offered through the district SACC program and (b) standard district after school care with homework help and no academic enrichment activity component.
Report Availability Knutson, K. A. (2003). CHAMPS SACC program evaluation report. Coral Springs, FL: Author.

Knutson, K. A. (2003). Evaluation of the CHAMPS after-school program: FY2003. Coral Springs, FL: Author.

Knutson, K. A. (2004). CHAMPS after-school program outcome evaluation year two: FY2004. Coral Springs, FL: Author.

Knutson, K. A. (2005). CHAMPS after-school program outcome evaluation year three: FY2005. Coral Springs, FL: Author.

Contacts

Evaluation Kimberly A. Knutson, Ed.D.
Specialist, Testing and Evaluation
School District of Palm Beach County
3370 Forest Hill Blvd., Suite 228
West Palm Beach, FL 33406-5871
Tel: 561-434-8782
Fax: 561-963-3842
Email: Knutson@palmbeach.k12.fl.us
Program Judy Nee
Director, CHAMPS Program
8487 155th Place North
Palm Beach Gardens, Florida 33418
Tel: 561-741-4641
Email: judynee@bellsouth.net
Profile Updated January 5, 2006

Evaluation 1: Evaluation of the CHAMPS After-School Program: FY2003


Evaluation Description

Evaluation Purpose To answer three questions: (a) Was the percentage of CHAMPS participants who scored at grade level or above on multiple standardized achievement measures greater than the percentage of comparable nonparticipants, (b) were the average spring standardized achievement test scores of CHAMPS participants greater than those of comparable nonparticipants, and (c) did CHAMPS participants achieve greater reading and math standardized achievement test score gains than comparable nonparticipants?
Evaluation Design Quasi-Experimental: The evaluation compared 478 second through fifth grade CHAMPS participants to 447 similar nonparticipants. The comparison group consisted of a random sample of youth stratified by grade, race/ethnicity, free/reduced-price lunch status (FRL), and limited English proficiency (LEP) status who were selected from a pool of youth not attending CHAMPS from each of the 10 school sites. The comparison group was matched proportionally to the program group on these demographic indicators. Subgroup analyses were done by grade and within grades for race/ethnicity, FRL status, and LEP status.
Data Collection Methods Secondary Source/Data Review: Achievement test score data were obtained from SDPBC.

Test/Assessments: The achievement test scores obtained from SDPBC consisted of the Florida Comprehensive Assessment Test of Sunshine State Standards (FCAT), the Florida Comprehensive Assessment Test-Norm Referenced Test (FCAT-NRT), and the Scholastic Reading Inventory (SRI). The FCAT is a criterion-referenced test that measures the degree to which students have mastered Florida’s standards for reading, writing, and math. The FCAT-NRT measures how well students perform in reading and math compared to a national sample of students. SRI scores measure reading comprehension. SRI scores are available beginning in second grade, while FCAT and FCAT-NRT scores are available beginning in third grade.

Using these test results, four measures were examined: (a) the percentage of youth performing at grade level on the FCAT, FCAT-NRT, and SRI in the spring; (b) average performance on the FCAT Developmental Scale (FCAT-DS), FCAT-NRT Normal Curve Equivalent (NCE), and SRI scores in the spring; (c) the amount of spring to spring gain on the FCAT-DS and FCAT-NRT, and the amount of fall to spring gain on the SRI; and (d) the effect size of statistically significant results for each analysis. (For test score gains, evaluators also calculated “effect size differences,” which are the differences in calculated pretest to posttest effect sizes for the treatment and comparison groups.) Effect size differences were considered meaningful if they reached the following thresholds: .20 = small difference, .50 = moderate difference; and .80 = large difference.
Data Collection Timeframe Data were collected for fiscal years 2002 and 2003.


Findings:
Summative/Outcome Findings

Academic The percentage of CHAMPS youth who scored above grade level did not differ significantly from the percentage of comparison group youth for any of the three tests, overall or for any subgroup.

The average reading SRI scores for CHAMPS second graders did not differ from those in the comparison group, overall or for any subgroup

White CHAMPS third graders had significantly lower average math FCAT and FCAT-NRT scores than their comparison group counterparts (p = .04 and p = .02, effect sizes of -.36 and -.40, respectively). No other significant differences were found for third graders’ math FCAT and FCAT-NRT scores, either overall or for any subgroup.

FRL CHAMPS fourth graders had significantly higher average reading and math FCAT scores than their comparison group counterparts (p = .03 and p = .05, effect sizes of .50 and .47, respectively). No other significant differences were found for fourth graders’ FCAT scores, either overall or for any subgroup.

CHAMPS fifth graders had significantly higher average SRI reading scores than their comparison group counterparts (p < .04, effect size of .27). This was also true for African American CHAMPS fifth graders (p < .01, effect size of .59). No other significant differences were found for fifth graders’ SRI scores, either overall or for any subgroup.

LEP CHAMPS third graders demonstrated smaller gains on the SRI than their comparison group counterparts (effect size difference = -.24). No other meaningful effect size differences were found for third graders’ SRI scores, either overall or for any subgroup.

CHAMPS fourth graders experienced larger FCAT reading score gains than their comparison group counterparts (effect size difference = .27). This was also true for CHAMPS fourth graders who were African American, FRL status, and LEP status (effect size differences = .40, .38, and .67, respectively).

CHAMPS fourth graders experienced smaller FCAT-NRT reading gains than their counterparts in the comparison group (effect size difference = -.21). African American CHAMPS fourth graders experienced smaller gains than their comparison group counterparts (effect size difference = -.55). Also, FRL CHAMPS fourth graders experienced a slight decline while their counterparts in the comparison group experienced gains (effect size difference = -.65).

On the SRI, no overall differences were found between the CHAMPS and comparison group fourth graders, but FRL and LEP CHAMPS fourth graders experienced greater gains than did their counterparts in the comparison group (effect size differences = .41 and .34, respectively).

No overall differences were found between CHAMPS and comparison group fourth graders for math scores on either the FCAT or the FCAT-NRT. However, CHAMPS fourth graders who were African American, FRL status, and LEP status all experienced greater FCAT gains than their comparison group counterparts (effect size differences = .41, .58, and .39, respectively), while African American CHAMPS youth and FRL CHAMPS fourth graders experienced smaller FCAT-NRT gains than their comparison group counterparts (effect size differences = -.30 each).

CHAMPS fifth graders experienced smaller FCAT reading score gains than their comparison group counterparts (effect size difference = -.23), while White CHAMPS fifth graders experienced smaller gains than their comparison group counterparts (effect size difference = -.27).

For fifth graders’ FCAT-NRT reading scores, no differences were found either overall or for any subgroup.

CHAMPS fifth graders demonstrated greater SRI score gains than comparison group youth (effect size difference = .27). This was also true for African American, White, and FRL subgroups (effect size differences = .43, .20, and .21, respectively).

CHAMPS fifth graders demonstrated greater gains in FCAT math scores than comparison group youth (effect size differences = .60). This was also true for the African American and FRL subgroups (effect size differences = 1.53 and .76, respectively). On the FCAT-NRT, no differences were found for fifth graders, either overall or for any subgroup.

© 2016 Presidents and Fellows of Harvard College
Published by Harvard Family Research Project