The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.
HFRP
> Out-of-School Time
> OST Database & Bibliography
> Database
> BELL Accelerated Learning Summer Program
> Evaluation #2 (2005-2006): Random Assignment Study
A Profile of the Evaluation of BELL Accelerated Learning Summer Program
Profile Information
- Full Text (HTML)
- How to cite
- About the OST Database
Program Description
| Overview | The BELL (Building Educated Leaders for Life) Accelerated Learning Summer Program (BELL Summer) is one of BELL’s two ongoing initiatives during nonschool hours—along with the BELL After School program. The BELL programs are held in four cities nationwide: Boston, New York, Baltimore, and Washington, DC. BELL’s initiatives are designed to increase children’s knowledge and mastery of reading, writing, and math; raise children’s academic expectations and self-esteem; empower parents; and develop effective mentoring relationships between children and positive adult role models. BELL Summer is a summer learning program that provides youth entering first through seventh grades with intensive academic instruction; hands-on educational, cultural, artistic, and recreational activities; guest speakers; community service projects; and field trips. |
| Start Date | 1992 |
| Scope | national |
| Type | summer/vacation |
| Location | urban |
| Setting | public school, other (college campus) |
| Participants | elementary and middle school students (entering Grades 1–7) |
| Number of Sites/Grantees | three sites (2002 and 2003); five sites (2005); six sites (2006) |
| Number Served | 305 in 2002; 1,350 in 2006 |
| Components | BELL Summer operates 8 hours a day, 5 days a week for 6 weeks. The program targets youth of color living in low-income urban communities who are performing below grade level in school. BELL targets youth from the BELL After School program who could benefit from summer programming. In addition, scholars (i.e., participants in BELL Summer) are recommended to the program by staff in BELL After School program schools, and scholars from previous BELL Summer years are invited to re-enroll. The program attempts to address children’s academic, social, and cultural needs as follows. Academic Skill Development. For scholars in both BELL After School and BELL Summer, skills-based literacy and math curricula aligned to state and national learning standards are taught by experienced, certified teachers and teachers’ assistants with a ratio of, at most, one adult for every eight children. Improved Confidence and Attitude Toward Learning. BELL Summer uses multicultural books, themes related to youth’s cultural heritages, and instruction according to youth’s needs, as identified by diagnostic tests and teacher assessments. Social/Community Skill Development. Community Time begins each program day and consists of activities and discussions to build a sense of community and supportive peer relationships. Guest speakers, who are typically representative of scholars’ race and ethnicity and/or are from their community, also share stories about their lives to inspire scholars about the possibilities for their own futures. Scholars participate in at least one community service project over the course of the summer program and are provided with cultural opportunities in art, music, dance, and drama, along with enriching and fun activities such as field trips. At the end-of-summer closing ceremony, scholars display their artwork and perform musical, dance, and dramatic productions that they have developed throughout the summer. BELL Summer also promotes parent engagement in academics. Parents are encouraged to read nightly with their children, attend program events, and keep reading logs with their children to track reading progress. |
| Funding Level | Approximately $2 million (2006 BELL Summer program budget); $15.9 million (BELL national operating budget) |
| Funding Sources | Bank of America, Boston Foundation, Charles Hayden Foundation, Citizens Bank Foundation, Comcast, Fidelity Investments, Highland Street Foundation, Jane’s Trust, Liberty Mutual, Lloyd G. Balfour Foundation, Lone Pine Foundation, Louis Calder Foundation, Robin Hood Foundation, Samberg Family Foundation, Smith Family Foundation, Starr Foundation, Weinberg Foundation |
Evaluation
| Overview | BELL undertakes evaluation to ascertain the extent to which its programs meet the needs of the children served, to improve its programs, and to disseminate best practices to the community. |
| Evaluator | BELL Duncan Chaplin, Mathematica Policy Research/The Urban Institute Jeffrey Capizzano, Teaching Strategies Inc. |
| Evaluations Profiled | BELL Accelerated Learning Summer Program 2002 National Evaluation Report Impacts of a Summer Learning Program: A Random Assignment Study of Building Educated Leaders for Life (BELL) |
| Evaluations Planned | BELL conducts evaluations at the conclusion of every BELL After School and BELL Summer program. |
| Report Availability | BELL. (2001). BELL Accelerated Learning Summer Program 2001 evaluation report. Dorchester, MA: Author. BELL. (2002). BELL Accelerated Learning Summer Program 2002 national evaluation report. Dorchester, MA: Author. BELL. (2003). BELL Accelerated Learning Summer Program: 2003 program outcomes. Dorchester, MA: Author. Chaplin, D., & Capizzano, J. (2006). Impacts of a summer learning program: A random assignment study of Building Educated Leaders for Life (BELL). Washington, DC: The Urban Institute. Available at www.urban.org/publications/411350.html. |
Contacts
| Evaluation | Tiffany M. Cooper Chief Program Officer BELL National 60 Clayton Street Dorchester, MA 02122 Tel: 617-282-1567 ext. 108 Fax: 617-282-2698 Email: tcooper@bellnational.org Duncan Chaplin Senior Research Methodologist The Urban Institute 2100 M Street, NW Washington, DC 20037 Tel: 202-261-5709 Email: paffairs@ui.urban.org |
| Program | Tiffany M. Cooper Chief Program Officer BELL National 60 Clayton Street Dorchester, MA 02122 Tel: 617-282-1567 ext. 108 Fax: 617-282-2698 Email: tcooper@bellnational.org |
| Profile Updated | November 7, 2006 |
Evaluation 2: Impacts of a Summer Learning Program: A Random Assignment Study of Building Educated Leaders for Life (BELL)
Evaluation Description
| Evaluation Purpose | To examine the effectiveness of BELL Summer in improving youth’s academic performance and behavior and parents’ behavior. |
| Evaluation Design | Experimental: Random assignment was used to determine which applicants were accepted at 3 of BELL’s 5 sites in 2005: 2 sites in Boston and 1 site in New York City. In total, 1,917 youth applied to BELL Summer at these 3 sites, more than double the 750 slots available (250 at each site). Of these applicants, 1,225 had signed parental permission to be in the study before being informed of whether or not they would be accepted into the program. No race or gender differences were found between those willing and not willing to participate in the study. Those who did not choose to participate in the study were not excluded in the lottery to decide who would be accepted to the program. Some of the youth with parental consent were excluded either for cost considerations (i.e., families with multiple children enrolled in BELL were more cost effective to study) or because randomization was inappropriate based on the number of program slots available at certain sites (i.e., randomization was not feasible because either no slots were available for some applicants, or there were more slots available than potential applicants). This left a total sample size of 1,087 applicants. Of these, complete youth test and survey data were available for 835 applicants for an overall weighted response rate of 78% for the control group and 78% for the program group. Parent survey response rates were somewhat lower at 73% and 70% for the control and program groups respectively. No significant differences were found between the program and control groups based on observed characteristics. |
| Data Collection Methods | Secondary Source/Data Review: Data on race, gender, and participation status of each applicant were obtained from the application forms the parents filled out and augmented with information from BELL staff on program participants. Surveys/Questionnaires: The parent surveys covered the types of learning activities the child engaged in during the summer of 2004 (briefly) and 2005 (in great detail), the child’s behaviors, parent involvement in youth’s educational development and other activities, background data, and parent activities. The youth surveys focused on the children’s perceptions of their academic skills. Tests/Assessments: The Gates-MacGinitie reading test, which measures reading skills and covers both vocabulary and comprehension, was administered to all children in the study. Youth entering Grades 3 to 7 were also administered the Perception of Ability Scale Score (PASS), which is a self-evaluation instrument appropriate for youth in Grades 3 and higher (Hay et al., 1997), measuring perceived ability in reading, math, and general learning. For those entering Grades 1 and 2, perceived ability was measured using the Academic Perceptions Inventory (API, Soares & Soares, 2000) in reading and arithmetic. The parent survey includes a subset of questions from the Social Skills Rating System (SSRS, Meisels, Atkins-Burnett, & Nicholson, 1996) that measure positive social skills (cooperation, assertion, self-control and responsibility) as well as problem behaviors such as aggressive acts, poor temper control, and sadness and anxiety. References: Hay, I., Ashman, A.F., & van Kraayenoord, C.E. (1997). Investigating the influence of achievement on self-concept using an intra-class design and a comparison of the PASS and SDQ-1 self-concept tests. British Journal of Educational Psychology, 67, 311–321. Meisels, S. J., Atkins-Burnett, S., & Nicholson, J. (1996). Assessment of social competence, adaptive behaviors, and approaches to learning with young children. Washington, DC: National Center for Education Statistics. Soares, L. M., & Soares, A. T. (2000). Academic perceptions inventory: Test manual/Advanced level. Trumbull, CT: Castle Consultants. |
| Data Collection Timeframe | Data were collected between August 2005 and January 2006. |
Findings:
Formative/Process Findings
| Recruitment/Participation | During the summer of 2005, BELL participants were less likely to be at home, in summer school, or in another program than nonparticipants (p < .01), though 19% of the control group was in another program and 17% was in summer school. The program group was also less likely to be with their parents and more likely to be with program staff. No differences were found between program and control youth on hours per week spent without someone over the age of 12. This last result also held when evaluators looked specifically at the weekday hours when the BELL program was in operation. |
Summative/Outcome Findings
| Academic | Statistically significant positive impacts on the Gates-MacGinitie reading test scores were found in children who attended BELL Summer (p < .05). When compared to students who attended the same number of school days, BELL Summer scholars had higher reading comprehension and overall reading scores. On average, parents in the control group reported that their children spent almost 12 hours per week in academic activities and read 6.6 books in July 2005. Being in the program group significantly increased both of these numbers by about 50%; hours per week in academic activities by 6.4 hours per week and books read by 3.9 (p < .01). Some parents of program youth reported that their children read as many as 50 books during the month. In order to participate in BELL, members of the program group had to reduce their time in other activities. The largest estimated significant impacts were found on time spent watching TV and playing computer games, doing chores, attending camp, attending other academic activities, hanging out with friends, using the Internet/computer, and attending cultural activities. Smaller but still significant reductions were found for participation in other academic programs; sports, music, or arts; volunteer work; and religious activities (all p < .01 except volunteer work and religious activities, which were significant at p < .05). No impacts were found for summer school or caring for other children. Estimated impacts on the two measures of academic self-concept were insignificant and fairly small. |
| Family | Significant impacts were found for parents encouraging their children to read and actually reading to their children (p < .05). Estimated program impacts on parents’ own summer education activities were not significant. However, parents of program group members had a 9% higher probability of taking computer classes than parents of control group members. |
| Youth Development | No effects were found for measures of children’s behavior. |