The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.
HFRP
> Out-of-School Time
> OST Database & Bibliography
> Database
> 21st Century Community Learning Centers—Owensboro Kentucky Public Schools
> Evaluation #2 (2001-2002): Year 2
A Profile of the Evaluation of 21st Century Community Learning Centers—Owensboro Kentucky Public Schools
Profile Information
- Full Text (HTML)
- How to cite
- About the OST Database
Program Description
| Overview | The Owensboro Public Schools (OPS) 21st Century Community Learning Centers (21st CCLC) project is an after school program for students in select public schools in Owensboro, Kentucky. The program, which consists of five Community Learning Centers that operate year round, began on August 1, 2000 and is scheduled to run through June 2003. The goal of the program is to provide safe, supervised, and fun learning opportunities for students in kindergarten through twelfth grade that serve to increase their academic skills, enhance their ability to interact positively with peers and adults, and reduce behavior problems. |
| Start Date | August 2000 |
| Scope | local |
| Type | after school, summer/vacation, before school, weekend, comprehensive services |
| Location | urban |
| Setting | public school |
| Participants | kindergarten through high school students |
| Number of Sites/Grantees | five schools |
| Number Served | 662 in 2000–2001 and 1,323 in 2001–2002 |
| Components | Program objectives include providing the following: (1) after school programs five days a week for students when school is not in session, (2) extended day/extended year academic assistance for students, (3) day care services when school is not in session, (4) parenting workshops in cooperation with existing community agencies, (5) increased access to school technology and media centers for students and parents, (6) age-appropriate recreation and enrichment activities for students in cooperation with community agencies, (7) research-based substance abuse prevention and intervention activities for students and families, (8) career and employment counseling for students and adults, and (9) expanded learning opportunities for Emotional or Behavioral Disorders (EBD) students for seven weeks when school is not in session. Each school's 21st CCLC center has a full-time Coordinator who handles the program's day-to-day management. Coordinators plan, organize, and help implement program activities, hire and supervise staff, communicate with parents, maintain an evaluation database, and collaborate with other school and community-based programs. A Project Director, who reports to the OPS Director of Personnel and 21st CCLC Advisory Council, oversees and supervises each site and provides an annual update to the U.S. Department of Education. Coordinators combined available resources with student interest data to begin programming. Coordinators also consulted with other resource representatives, such as Family Resource/Youth Service Center (FRYSC) staff, teachers, parents, and other after school programs to determine what services were already available and what services were needed. The initial focus was on developing high-interest programming to attract students, followed by programs related to other areas. Programs are offered in areas of literacy, math, science, humanities, sports/games, technology, community service, culture, health, youth development, and adult services. Examples of programs offered at the Foust site include: community-led activities, Computer Wizards, Enrichment Clubs, Fun N' Fitness, Fun N' Fitness/Dream Team, Girl Scouts, humanities, Phonemic Awareness, prevention activities, and targeted tutoring. Examples of programs offered at the Estes site include: academic enrichment, community service, Is That Your Final Answer?, social/behavioral development, summer activities, and technology. The 5/6 Center site offers learning lab, model builders club, newspaper, and STAR. Examples of programs offered at the Owensboro middle school site include: Boy Scouts, Champions Against Drugs, Choices, craft class, Gents, lock-in at the YMCA, photography, recreation, technical school, tutoring, and We Care. Examples of programs offered at the Owensboro High School site include: arts and crafts, chess, Christmas Cookery, Computer Fun, Cooking with Meschko, drawing and sketching, employment search, knitting circle, Mad Potters, Manuel's Muralists, photography, Raku Fest, Sew Much More, teen support group, and tutoring. Students were able to choose activities. |
| Funding Level | Total funding from the US Department of Education is $1,856,740 and total nonfederal funding is $2,191,742 (August 2000 through June 2003). |
| Funding Sources | U.S. Department of Education |
Evaluation
| Overview | The Owensboro Public School system contracted with REACH of Louisville, Inc. to conduct a systematic assessment of the Owensboro Public School system contracted with REACH of Louisville, Inc. to conduct a systematic assessment of the 21st CCLC program for each year of implementation. The overarching evaluation purposes are to: (1) develop an evaluation framework for organizing and gathering data, (2) create a computer software application to “house” program data and facilitate program evaluation, (3) gather, aggregate, and analyze program data to provide program staff, key stakeholders, community members, and others information about the overall implementation and effectiveness of the program, and (4) make recommendations for ongoing program development and improvement. A logic model outlining the project theory was developed and served as a guide for determining implementation benchmarks and interim outcomes, which were then tracked and analyzed over time to explore program processes and degree of implementation. Several outcomes were developed to evaluate the success of the program. Specifically, by the end of the three years of operation, the following should be achieved:
|
| Evaluator | Robert J. Illback, Psy.D. and Ben W. Birkby, Psy.D. of REACH of Louisville, Inc. |
| Evaluations Profiled | Formative Evaluation of the 21st Century Community Learning Centers, Year 1 Evaluation of the 21st Century Community Learning Centers Program, Year 2 |
| Evaluations Planned | The year 3 evaluation will be summative in nature, focusing on long-term distal program outcomes (e.g., academic achievement, reduction in suspensions). |
| Report Availability | Illback, R. J., & Birkby, B. W. (2001). Formative evaluation of the 21st Century Community Learning Centers, year 1. Louisville, KY: REACH of Louisville. Birkby, B. W., & Illback, R. J. (2002). Evaluation of the 21st Century Community Learning Centers program, year 2. Louisville, KY: REACH of Louisville. |
Contacts
| Evaluation | Robert J. Illback, Psy.D. Benjamin Birkby, Psy.D. REACH of Louisville, Inc. 101 East Kentucky Street Louisville, KY 40203 Tel: 502-585-1911 Fax: 502-589-1592 Email: illbackr@reachoflouisville.com (Robert J. Illback) birkbyb@reachoflouisville.com (Benjamin Birkby) |
| Program | 21st Century Community Learning Centers Owensboro Public Schools 1335 West 11th Street Owensboro, KY 42301 Tel: 270-686-1000 Fax: 270-686-5756 |
| Profile Updated | June 20, 2003 |
Evaluation 2: Evaluation of the 21st Century Community Learning Centers Program, Year 2
Evaluation Description
| Evaluation Purpose | To answer the following questions about the initial year of program implementation: (1) what are current enrollment, gender, and racial characteristics across sites? (2) what are current referral characteristics of students enrolled across sites? (3) what types of activities were offered? and (4) what proximal outcomes (e.g., satisfaction with program and perceptions of program benefits) are occurring? how do they compare to year 1? |
| Evaluation Design | Non-Experimental: The research sample included students, parents, teachers, program staff, and key informants involved in the five Community Learning Centers in Owensboro High School, Owensboro Middle School, Owensboro 6/5 Center, Foust Elementary School, and Estes Elementary School. |
| Data Collection Methods | Document Review: Evaluators examined literature and written communication related to the programs. Interviews/Focus Groups: Semi-structured interviews were conducted with parents (n=13), students (n=18), and key informants (n=43) at each location during a two-day visit in May 2002. Key informants interviewed were a representative sample of individuals who were familiar with the program, and included principals, teachers, FRYSC coordinators, 21st CCLC program staff, and selected others (e.g., school counselors). Three site coordinators and the former Project Director were interviewed at a later date by phone. Interviews examined the following aspects of the program: program evolution, program presence, comprehensiveness of services, referral and engagement process, community involvement, program accomplishments, program barriers, parent involvement, and continued program accessibility. Observation: Evaluators observed programs over a two-day period in May 2002. Secondary Source/Data Review: Student referral needs data were examined. Referral data included factors that led to students being assessed as needing 21st CCLC services, including such factors as academic underachievement and discipline or behavior problems. Referrals came from parents, teachers, and occasionally from students themselves. Coordinators completed initial referral and tracking forms for each student that included information about the selection/engagement strategy used to attract or recruit the student, barriers or problems encountered in program implementation, reasons for referral of the student to the program, and information about the student's program attendance patterns. School discipline records were also analyzed. Surveys/Questionnaires: Students, teachers, and parents were administered surveys. Teachers were surveyed at 3, 6, 9, 12, 15, 18, and 21 months after a student had entered the 21st CCLC program, and were asked to report whether or not students in their classes (who were involved in the 21st CCLC) had improved, not improved, or experienced no change in academic or behavioral progress. A total of 472 useable teacher surveys were administered during year 2, an improvement from year 1 in which 147 teacher surveys were collected. Of the 472 surveys collected during year 2, 386 individual students were represented. Sixty-seven surveys were administered at three months, 332 were administered at six months, 40 were administered at nine months, 9 were administered at 12 months, 12 were administered at 15 months, 7 were administered at 18 months, and 5 were administered at 21 months. Because the six-month timeframe included the largest data set (332 individual students), it was chosen to represent teacher perception of student functioning after substantial involvement in the 21st CCLC program. A total of 293 student surveys were administered during year 2 (24% of the total student enrollment across sites). Students were asked about their satisfaction with the program. A total of 137 parent surveys were administered during year 2. While this number is up from 119 parent surveys in year 1, the response rate is lower (11% vs. 18%) than in year 1 due to increased program enrollment. Parents were asked about their satisfaction with the program and their perception of any program benefits to their children. |
| Data Collection Timeframe | Data were collected during the 2001–2002 school year. |
Findings:
Formative/Process Findings
| Activity Implementation | Program sites increased the number of educational enrichment activities offered during year 2 and increased the total number of activities offered overall. Informal analysis from the Annual Performance Report completed by the former Program Director indicated that sites offering primarily educational enrichment activities (compared to primarily child care services) enjoyed better success and retention of students. When asked what the best things about the program were, students most frequently mentioned, “Getting to choose what activities I participate in.” Other responses included catching up on homework, “learning stuff,” being around 21st CCLC staff, and participating in high-appeal activities. Parents remarked that the program offered a broader range of activities in year 2 than in year 1, demonstrated better organization, and offered an adequate and appropriate range of activities due to the good variety of types of activities offered, and attempts to incorporate student input. |
| Parent/Community Involvement | When asked how the program had tried to involve them, parents most often responded that their involvement was peripheral (similar to year 1). Parents were typically kept informed (via phone calls, materials sent home, face to face contact) of the program's activities and their child's progress, but were less often involved in joint activities with their children (although more occurred in year 2 than in year 1). This occurred for several reasons, including parents' work schedules interfering with after school programming, few parent-activities offered, and sometimes transportation barriers. Several parents noted that the 21st CCLC had an open door policy and all parents felt comfortable talking with program staff. Barriers to parent involvement noted by key informants were financial, work, and transportation struggles, as well as communication barriers (e.g., no phone), and issues related to low-parent interest. |
| Program-School Linkages | Many key informants noted that program “presence” in the schools grew during year 2. Many perceived improved communication and coordination with existing school programs, as well as an array of services and activities appropriate and sufficient to meet the needs of students, as reasons for this improved presence. A few informants reported struggling with lack of support from school administration (e.g., principals). |
| Recruitment/Participation | Forty-seven percent (1,323) of the total school population across all five sites (2,794) was enrolled in the 21st CCLC program during year 2, up from 24% in year 1. In addition, the program exceeded its goal of serving 1,230 students per year. Similar to year 1, the program serves roughly equal numbers of females and males. Racial and ethnic characteristics of program participants were quite similar to year 1, with 74% of participants identifying as Caucasian, 21% identifying as Black/African American, 4% identifying as biracial, and less than 1% in each of the remaining three categories, Hispanic/Latino, Asian, and American Indian/Alaskan. Data from program referral forms indicated that 41% of 21st CCLC participants came to the program due to being negatively affected by poverty, 36% had limited access to social and recreational interactions with peers, 29% were experiencing boredom or lack of opportunities for constructive activities, 28% experience limited access to technology resources, 20% were considered to be underachieving academically, 12% were experiencing repeated discipline referrals and behavior problems, 10% were considered to be quiet or withdrawn, and 5% were rated as fearful, anxious, or afraid. (Note that more than one referral characteristic can be marked so the total is greater than 100%). During year 2, 23% of students (310 out of 1,323) attended more than 30 days of activities (classifying them as “regular attendees”). This is down from the 51% of year 1 students (340 out of 662) who were regular attendees. Parents remarked that they became aware of the 21st CCLC program through booths set up during school registration, flyers sent home with children, and through their children becoming aware of the program through the involvement of other students. Some parents mentioned a lack of targeted recruitment of students that might potentially benefit from the program as an area for improvement. Key stakeholders at the middle and high school agreed that this would help improve the referral and engagement process. Several site coordinators mentioned that building better relationships with the school counselor and other key members of the existing school staff was necessary to improve the referral and engagement process. Several site coordinators noted that Intercessions were a great place for marketing their program and for recruiting students. Students noted that word of mouth was the most common way students became aware of the 21st CCLC program. In explaining why some students don't participate, students mentioned (a) lack of transportation home, (b) other activities students might be involved in, (c) a possible negative stigma (e.g., troubled kids, unpopular kids) associated with the program, and (d) perception that the program might be boring or “more schoolwork.” |
| Satisfaction | Students continued to be highly satisfied with the program, with 98% reporting they “like it,” 94% saying they look forward to coming, and 96% saying they feel comfortable and that someone is available to help them at the program. Ninety-four percent of surveyed parents reported being “very satisfied” with the program, while the remaining 6% reported being “satisfied.” Parents unanimously perceived the program to be accessible and “easy to use,” given the transportation provided by the program, the flexibility of program staff, and the collaboration between the 21st CCLC program and other programs offered at the school, such as the Family Resource Center. |
| Staffing/Training | The entire middle school staff resigned in early spring 2002. Many key informants noted that this created a positive shift in the philosophy and “culture” of the 21st CCLC, although this situation was also noted to be a program barrier when it initially occurred. |
Summative/Outcome Findings
| Academic | Seventy-seven percent of surveyed students reported that they were doing better at school because of the 21st CCLC program. Parents continued to see academic benefits to their children due to the 21st CCLC program, with surveyed parents reporting that their children look forward to school (88%) and do better in school (84%) because of the program. At the six-month timeframe, approximately 50% to 60% of teachers reported that 21st CCLC students had improved in turning in homework, completing homework, participating in class, regular attendance, attentiveness in class, academic performance, and preparedness for class. Very few teachers noted improvements in school suspensions due to the program. |
| Family | Seventy-four percent of surveyed parents reported that the program helped their family have less stress. |
| Workforce Development | Sixty-seven percent of surveyed parents reported that the program helped them remain employed. Twenty-four percent reported that the program helped them remain in school. |
| Youth Development | At the six-month timeframe, approximately 50% to 60% of teachers reported that 21st CCLC students had improved in volunteering, being well behaved, and getting along well with others. Very few teachers reported improvements in referrals to the office due to the 21st CCLC program. |