Search Publications | The Evaluation Exchange | FINE Network | OST Database

The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.

A Profile of the Evaluation of 21st Century Community Learning Centers—District of Columbia

Program Description

Overview

The District of Columbia 21st Century Community Learning Center (DC 21st CCLC) program began in 1999 as part of a broader initiative, the DC Children and Youth Investment Partnership (DC CYIP), which aims to coordinate youth activities in the District of Columbia so that all youth receive a “seamless web” of services that help them grow up to be healthy and productive adults. The DC 21st CCLC was run by the District of Columbia Public School System (DCPS) and offered school-based after school, summer, and Saturday programs for youth and evening classes for adults. The DC 21st CCLC's three major goals were to: (1) offer significant, expanded learning opportunities for children, youth, and adults in the local school community; (2) help middle school youth meet or exceed state standards in reading and mathematics; and (3) reduce substance abuse and teen violence.

Start Date	fall 1999 (completed summer 2002)
Scope	local
Type	after school, summer/vacation, comprehensive services
Location	urban
Setting	public schools
Participants	kindergarten through middle school students
Number of Sites/Grantees	Ten school sites. DCPS is the grantee, receiving funds directly from the U.S. Department of Education. DCPS, in turn, funded 10 public junior high and middle schools in Washington, D.C. starting during the 1999–2000 school year and going though to 2001–2002. In general fewer sites operated during the summer. For instance, only 9 schools were funded for the summer program in 2000. A few of these schools combined programs and some programs shared sites so that there were a total of only five sites. During the summer of 2001, programming operated at 8 of the 10 21st CCLC school sites. The program was closed during the summer of 2002 due to funding constraints. The after school program took place at all 10 middle/junior high schools during the regular school years from the spring of 2000 to the spring of 2002.
Number Served	1,371 in the after school program during 1999–2000; at least 989 children in the summer program in 2000; 920 children in the summer program in 2001
Components	Each DC 21st CCLC site was to deliver offerings in each of the following areas: (1) education (reading, math, and technofluency); (2) sports/health (physical and mental well-being); (3) arts (visual and performing); and (4) community service. Each DC 21st CCLC summer program was located in a public school and managed by an Assistant Principal (AP) who was responsible for a staff of instructors called facilitators. The program operated from 12:30pm to 6:30pm each weekday during the summer of 2000 and in the morning and afternoons during the summer of 2001. The APs and staff from DCPS collaboratively plan the programs. During the summer of 2001, an academic summer school program, which involved the use of computers in many activities, was offered in the mornings and a nonacademic (e.g., sports, arts, community service, and technology) program was offered in the afternoon. The DC 21st CCLC after school program was established with a flexible design for program implementation. Within the DC 21st CCLC framework, APs worked with school staff, students, and community members to identify and deliver activities that served the needs and interests of the local student population. Most APs selected DCPS teachers as facilitators to conduct specific student activities at their sites. APs were also charged with involving local businesses and community leaders in planning and/or implementing after school programming, as well as with forming a Neighborhood Advisory Council to provide community input on the 21st CCLC programming.
Funding Level	approximately $15 million over three years, according to original plans
Funding Sources	Federal government. DCPS received $4.1 million from the US Department of Education through the 21st CCLC Program. Most of the remaining funds came from the Temporary Assistance for Need Families (TANF) block grant.
Other	The DCPS provided training to APs prior to program implementation and then reviewed implementation with them in meetings and special workshops once the program began. Trainings and meetings covered such topics as the technology and sports program components, how to document program activities, and how to manage and train staff. There were also DCPS trainings held for facilitators regarding the philosophy of the program, activities to be conducted, and specific support for sports coordinators..

Evaluation

Overview	The Urban Institute conducted formative evaluations of the DC 21st CCLC program that focused on implementation and the identification of possible student outcome areas to guide future evaluation work. One evaluation focused on the after school program during the 1999–2000 school year, a second on the summer program during the summer of 2000, and a third on the summer program during 2001 (with a particular focus on the use of computer technology to improve academic achievement).
Evaluators	Jacqueline Raphael, Duncan Chaplin, Zakia Redd, Luke Miller, Meredith Liu, Victoria Russell, Helen Fu, and Emily Anthony from the Urban Institute
Evaluations Profiled	Formative Report on the DC 21st Century Community Learning Center After-School Program Formative Report on the District of Columbia 21st Century Community Learning Center Summer Program Using Technology to Improve Academic Achievement in Out-of-School-Time Programs in Washington, DC
Evaluations Planned	An impact evaluation should be completed by October 2003.
Report Availability	Raphael, J., Chaplin, D., & Redd, Z. (2000). Formative report on the District of Columbia 21st Century Community Learning Center summer program. Washington, DC: The Urban Institute. Available at: www.urban.org/url.cfm?ID=409651. Raphael, J., Chaplin, D., Miller, L., & Redd, Z. (2000). Formative report on the DC 21st Century Community Learning Center after-school program. Washington, DC: The Urban Institute. Available at www.urban.org/url.cfm?ID=410440. Liu, M., Russell, V., Chaplin, D., Raphael, J., Fu, H., & Anthony, E. (2002). Using technology to improve academic achievement in out-of-school-time programs in Washington, DC Washington, DC: The Urban Institute. Available at www.urban.org/url.cfm?ID=410578.

Contacts

Evaluation	Jacqueline Raphael The Urban Institute 2100 M St., NW Washington, DC 20037-1207 Tel: 202-261-5809 Fax: 202-833-2477 Email: jac.8@juno.com	Duncan Chaplin The Urban Institute 2100 M St., NW Washington, DC 20037-1207 Tel: 202-261-5771 Fax: 202-833-2477 Email: dchaplin@ui.urban.org
Program	Saundra Handy Shaw JHS 925 Rhode Island Ave., NW Washington, DC 2002 Tel: 202-607-4208 Fax: 202-673-2463 Email: handy_s4@yahoo.com
Profile Updated	August 18, 2003

Evaluation 1: Formative Report on the DC 21st Century Community Learning Center After-School Program

Evaluation Description

Evaluation Purpose	To describe the implementation of the DC 21st CCLC after school program between October 1999 and May 2000 and the implications of the current implementation for further evaluation of the program.
Evaluation Design	Non-Experimental: Data were collected from all individual after school program sites, including APs and students.
Data Collection Methods	Document Review: DCPS' grant proposal to the US Department of Education for the DC 21st CCLC program, as well as each of the 10 schools' after school program grant proposal to DCPS were analyzed. Also reviewed was the DCPS Performance Report for the program, submitted to the US Department of Education in April 2000. Interviews/Focus Groups: Semi-structured interviews were conducted with the 10 APs, which lasted about one hour and covered issues of program implementation at the sites. Semi-structured focus groups were also conducted with between seven and twelve students at each site, which lasted about 30 minutes and covered issues of students' participation and their impressions of the program, including any potential benefits to them from their participation. Observation: Informal observations of program facilities and students' participation in program activities were made to provide information about program implementation.
Data Collection Timeframe	Data were collected during the 1999–2000 school year.

Findings:
Formative/Process Findings

Activity Implementation	Due to Department of Education oversight rules prohibiting access to funds until late February 2000, many after school activities could not be implemented until that time; these include: direct instruction in reading and mathematics, development of technology and academic skills on DC 21st CCLC computers, substance abuse, pregnancy, and violence prevention activities, intergenerational activities, and educational and entertainment field trips. Despite these limitations, nearly all sites reported student activities being conducted in all four key component areas (education, sports/health, arts, and community service) prior to this date, although the intensity and duration of such activities varied widely across the 10 program sites. Most sites offered activities in all four components every week or two. The evaluators found that a wide variety of activities were offered across the 10 sites, indicating that APs were tailoring program offerings to local needs and staff capabilities. APs frequently mentioned that students' interests were a significant factor in deciding what activities to offer. At most sites, students participated in two or three activities per day. Several programs gave students a “choice” day, typically Fridays, when students were able to select the activities in which they wished to participate. The degree of student choice varied considerably from site to site.
Parent/Community Involvement	APs reported moderate levels of collaboration with two or more community entities, including neighborhood businesses, churches, religious groups, foundations, sports facilities, and various guest speakers. Collaboration usually entailed donated services, facilities, staffing, and/or transportation to the sites. At the “high end” of community involvement, one site had local business owners teach students entrepreneurial skills, while students at another site received individual tutoring from local college student volunteers. All but one site reported having formed Neighborhood Advisory Councils, but none had yet conducted meetings at the time of the site visits. Many APs noted it was difficult to find meeting times that accommodated the majority of members' schedules. APs and students reported that parents were not yet very involved in the after school programs, but several APs reported having plans to engage more parents.
Program Context/Infrastructure	APs implemented various policies for maintaining discipline and student safety in the program, including: suspending disruptive students from the program, employing security guards and metal detectors, requiring written parental permission for students to participate, making sure nonparticipants didn't “hang out” in the school or on the fields during after school program operation, and locking doors during the program. Students at several schools mentioned that the program helps to keep them off the streets. Many said that if they were not in the program they would be hanging out on the streets, doing “nothing,” watching television, sleeping, or participating in unsupervised recreational or academic activities.
Recruitment/Participation	APs publicized the program by making announcements over the school intercom, posting notices in the school, talking about the program at PTA and YMCA meetings, sending home flyers/letters/brochures, and soliciting referrals from guidance counselors. The program reached almost 70% of its overall enrollment goal of serving 2,000 students across all sites, but routinely served less than half this number on a day-to-day basis. APs suggested that family needs, distance between students' homes and school, lack of initial funding (inhibiting the provision of popular activities), other after school activities, and lack of student interest may have inhibited greater student participation.
Satisfaction	Students reported that they enjoyed the program and participated in activities to which they would not otherwise have access. Students were especially enthusiastic about the sports activities offered, and the individualized academic attention they received from facilitators. Students expressed positive feelings in regards to their interactions with peers at the program, especially opportunities to interact with students from other grade levels.
Staffing/Training	Seven out of 10 APs reported having worked in some capacity at their sites for at least 10 years, three of them between 17 and 30 years. Several reported having worked at after school programs that existed before the 21st CCLC was initiated. Most APs personally recruited program facilitators. They sought staff with flexible skills and the ability to nurture and motivate students. Several APs specifically recruited facilitators with skills in areas such as computer literacy and mathematics. Many students reported meaningful interactions with the adult program staff. Students reported especially enjoying interaction with the staff in a non-classroom setting, such as playing sports. A number of APs expressed concerns about how their roles were defined at the school, particularly in regard to their responsibilities in relation to the regular school day vs. the after school programming. Some expressed frustrations related to insufficient time to devote attention to the after school program, lack of program assistants, and redundant paperwork required for the 21st CCLC program. Some APs expressed concerns about facilitator staffing, in that some facilitators were sometimes late due to regular day positions at other schools. Some APs wanted to hire facilitators from outside the DCPS system, but were advised not to by DCPS because of the extra time needed to process non-DCPS applicants, leading some APs to feel limited in their hiring powers. Some APs reported difficulty dealing with the 1:20 staffing ratio, given that program attendance fluctuated quite a bit on a daily basis. APs reported some staff had difficulty managing students and maintaining discipline. Both students and APs indicated a need for more staff to ensure student safety, particularly in getting students home safely from the program.

A Profile of the Evaluation of 21st Century Community Learning Centers—District of Columbia

Profile Information