You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.

www.HFRP.org

The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.

Terms of Use ▼


Program Description

Overview The BELL (Building Educated Leaders for Life) Accelerated Learning Summer Program (BELL Summer) is one of BELL’s two ongoing initiatives during nonschool hours—along with the BELL After School program. The BELL programs are held in four cities nationwide: Boston, New York, Baltimore, and Washington, DC. BELL’s initiatives are designed to increase children’s knowledge and mastery of reading, writing, and math; raise children’s academic expectations and self-esteem; empower parents; and develop effective mentoring relationships between children and positive adult role models. BELL Summer is a summer learning program that provides youth entering first through seventh grades with intensive academic instruction; hands-on educational, cultural, artistic, and recreational activities; guest speakers; community service projects; and field trips.
Start Date 1992
Scope national
Type summer/vacation
Location urban
Setting public school, other (college campus)
Participants elementary and middle school students (entering Grades 1–7)
Number of Sites/Grantees three sites (2002 and 2003); five sites (2005); six sites (2006)
Number Served 305 in 2002; 1,350 in 2006
Components BELL Summer operates 8 hours a day, 5 days a week for 6 weeks. The program targets youth of color living in low-income urban communities who are performing below grade level in school. BELL targets youth from the BELL After School program who could benefit from summer programming. In addition, scholars (i.e., participants in BELL Summer) are recommended to the program by staff in BELL After School program schools, and scholars from previous BELL Summer years are invited to re-enroll.

The program attempts to address children’s academic, social, and cultural needs as follows.

Academic Skill Development. For scholars in both BELL After School and BELL Summer, skills-based literacy and math curricula aligned to state and national learning standards are taught by experienced, certified teachers and teachers’ assistants with a ratio of, at most, one adult for every eight children.

Improved Confidence and Attitude Toward Learning. BELL Summer uses multicultural books, themes related to youth’s cultural heritages, and instruction according to youth’s needs, as identified by diagnostic tests and teacher assessments.

Social/Community Skill Development. Community Time begins each program day and consists of activities and discussions to build a sense of community and supportive peer relationships. Guest speakers, who are typically representative of scholars’ race and ethnicity and/or are from their community, also share stories about their lives to inspire scholars about the possibilities for their own futures. Scholars participate in at least one community service project over the course of the summer program and are provided with cultural opportunities in art, music, dance, and drama, along with enriching and fun activities such as field trips. At the end-of-summer closing ceremony, scholars display their artwork and perform musical, dance, and dramatic productions that they have developed throughout the summer. BELL Summer also promotes parent engagement in academics. Parents are encouraged to read nightly with their children, attend program events, and keep reading logs with their children to track reading progress.
Funding Level Approximately $2 million (2006 BELL Summer program budget); $15.9 million (BELL national operating budget)
Funding Sources Bank of America, Boston Foundation, Charles Hayden Foundation, Citizens Bank Foundation, Comcast, Fidelity Investments, Highland Street Foundation, Jane’s Trust, Liberty Mutual, Lloyd G. Balfour Foundation, Lone Pine Foundation, Louis Calder Foundation, Robin Hood Foundation, Samberg Family Foundation, Smith Family Foundation, Starr Foundation, Weinberg Foundation

Evaluation

Overview BELL undertakes evaluation to ascertain the extent to which its programs meet the needs of the children served, to improve its programs, and to disseminate best practices to the community.
Evaluator BELL

Duncan Chaplin, Mathematica Policy Research/The Urban Institute

Jeffrey Capizzano, Teaching Strategies Inc.
Evaluations Profiled BELL Accelerated Learning Summer Program 2002 National Evaluation Report

Impacts of a Summer Learning Program: A Random Assignment Study of Building Educated Leaders for Life (BELL)
Evaluations Planned BELL conducts evaluations at the conclusion of every BELL After School and BELL Summer program.
Report Availability BELL. (2001). BELL Accelerated Learning Summer Program 2001 evaluation report. Dorchester, MA: Author.

BELL. (2002). BELL Accelerated Learning Summer Program 2002 national evaluation report. Dorchester, MA: Author.

BELL. (2003). BELL Accelerated Learning Summer Program: 2003 program outcomes. Dorchester, MA: Author.

Chaplin, D., & Capizzano, J. (2006). Impacts of a summer learning program: A random assignment study of Building Educated Leaders for Life (BELL). Washington, DC: The Urban Institute. Available at www.urban.org/publications/411350.html.

Contacts

Evaluation Tiffany M. Cooper
Chief Program Officer
BELL National
60 Clayton Street
Dorchester, MA 02122
Tel: 617-282-1567 ext. 108
Fax: 617-282-2698
Email: tcooper@bellnational.org

Duncan Chaplin
Senior Research Methodologist
The Urban Institute
2100 M Street, NW
Washington, DC 20037
Tel: 202-261-5709
Email: paffairs@ui.urban.org
Program Tiffany M. Cooper
Chief Program Officer
BELL National
60 Clayton Street
Dorchester, MA 02122
Tel: 617-282-1567 ext. 108
Fax: 617-282-2698
Email: tcooper@bellnational.org
Profile Updated November 7, 2006

Evaluation 1: BELL Accelerated Learning Summer Program 2002 National Evaluation Report



Evaluation Description

Evaluation Purpose To answer the following questions: (a) Was program involvement associated with an increase in scholars’ reading skills, math skills, and self-concept; (b) did scholars develop social/community skills; and (c) what did parents and scholars think of the program?
Evaluation Design Quasi-Experimental: Evaluators collected pre- and post-outcome measures from all youth enrolled across the 3 sites using standardized assessment tools so that results could be compared to national norms on these assessments. They also compared BELL Summer scholars’ learning changes to summer learning changes for typical low-income youth, as reported by Cooper, Nye, Charlton, Lindsay, and Greathouse (1996).

Reference:
Cooper, H., Nye, B., Charlton, K., Lindsay, J., & Greathouse, S. (1996). The effects of summer vacation on achievement test scores: A narrative and meta-analytic review. Review of Educational Research, 66, 227–268.
Data Collection Methods Document Review: Evaluators reviewed “portfolios” for each scholar, which include writing samples, book reports, quizzes, and math and literacy exercises collected in the program. These portfolios also contain records of each scholar’s individual goals, their teachers’ and parents’ assessment of their weaknesses, and teaching staff’s observations of their individual progress. For 6-week programs, teachers completed progress reports based on each scholar’s portfolios after 3 weeks and again at the end of 6 weeks. Teachers at the 4-week program completed just one progress report at the end of the program.

Secondary Source/Data Review: Family income data from scholars at the Boston sites were examined.

Surveys/Questionnaires: The Self-Concept Survey, developed by BELL, required scholars to describe who they are academically and socially and where they feel their weaknesses and strengths lie. The pretest survey asked scholars to supplement their writing with a drawing that demonstrated “what school was like last year.” The writing sample and drawing served as a baseline measure of scholars’ self-concept. Scholars were asked again on the final day of the program to respond to the same survey, as a means of reflecting on their experience and to indicate changes in competency and/or self-concept. The posttest survey asked scholars to draw what they think “school will be like next year.” The evaluators used a scoring rubric to assess the content and affect of the responses. There were two versions of the self-concept measure. One version asked scholars in Grades 3 to 6 to respond to four open-ended questions and complete a drawing exercise. The second version asked scholars in Grades 1 and 2 to complete only the drawing exercise.

The BELL Summer Report Card is BELL’s survey for parents. At the end of the program, parents were asked to grade BELL Summer regarding its structure and content, program staff, effectiveness at reaching objectives for scholars and at partnering with parents. They were also asked to grade the overall program as an indication of their overall satisfaction.

Tests/Assessments: The Stanford Diagnostic Reading (SDRT-IV) test was used as a pre- and post-measure of scholars’ reading skills at the third and fourth grade levels (scholars entering third and fourth grade in the fall). In Washington, DC, where the sample size was substantially smaller, all scholars, regardless of grade level, were administered the Reading Test as a pretest and posttest. Scholars in the 6-week programs (Boston and New York) at the second, fifth, sixth, and seventh grade levels were also administered the Stanford Diagnostic Math Test (SDMT-IV) as a pre- and post-measure of scholars’ math skills. Scholars at the first grade level lacked necessary literacy and test-taking skills to complete either standardized test (BELL Summer first graders had just completed kindergarten). Stanford Performance Levels range from “failing” to “advanced” and indicate attainment of national benchmarks. Scores were also reported as grade equivalent (GE) scores, which are scores showing a youth’s progress in terms of typical youth’s scores at particular grade levels.
Data Collection Timeframe Data were collected during the summer of 2002.


Findings:
Formative/Process Findings

Recruitment/Participation In 2002, 25% of the 305 BELL Summer scholars came from the 2001–2002 BELL After School program.

The average family income of BELL Summer scholars at the Boston site was $18,729.

Eighty-nine percent of all scholars were below grade level in reading according to their GE scores, and 83% were below grade level in math. On average, scholars began BELL Summer one full grade level behind in reading and 9 months behind in math.

On average, scholars attended 26 of the 30 scheduled program days. The median attendance was 27 days, and 22% of scholars had perfect attendance.
Satisfaction The BELL Summer Report Card indicated that parents gave the program a B+ in all areas. Parents’ additional comments on the report card revealed feelings that the program helped their children and that parents’ valued the services provided by BELL Summer.


Summative/Outcome Findings

Academic Significant gains were seen in both reading and math for 6-week program participants (p < .001) and in reading for the 4-week program participants (p < .05).

For the two 6-week programs, the average changes in GE scores were a 4-month gain in reading and a 3-month gain in math. For the 4-week program, fourth graders demonstrated little GE improvement on average, but the second and third graders experienced approximately 6 months worth of GE improvement.

Second grade BELL Summer scholars gained 6 months in GE scores in reading, as compared to gains of approximately 1 month for typical low-income youth. Third grade BELL Summer scholars gained approximately 3 months in GE reading scores, as compared to losses of about 1 month for typical low-income youth. Finally, fourth grade BELL Summer scholars gained approximately 2 months in GE reading scores, as compared to losses of about 3 months for typical low-income youth.

Seventeen percent of BELL Summer scholars improved one SDRT-IV level in reading over 6 weeks, and 21% improved one SDRT-IV level in math. In the 4-week program, 5% moved to a higher category.

Sixty-two percent of scholars in the 6-week programs and 17% of scholars in the 4-week program who were at the “needs improvement” level in reading according to the SDRT-IV scale moved up to the “proficient” level by the end of the program.

Almost 30% of scholars in the 6-week programs who were at the “needs improvement” level in math according to the SDRT-IV scale moved up to the “proficient” level by the end of the program.

Progress reports indicated that 91% of scholars demonstrated improvement in writing related portfolio products, and 74% each in reading and math related portfolio products.
Youth Development At the end of the program, more scholars reported competencies in both social and academic domains—as opposed to just one domain —than at the beginning of the program. When asked what they were “good at” during the first week of the program, 38% indicated both social and academic areas (44% indicated social areas alone, and 16% indicated academic areas alone). When asked again during the final week, 64% indicated both social and academic competencies (27% indicated social areas alone, and 9% indicated academic areas alone).

Evaluation 2: Impacts of a Summer Learning Program: A Random Assignment Study of Building Educated Leaders for Life (BELL)



Evaluation Description

Evaluation Purpose To examine the effectiveness of BELL Summer in improving youth’s academic performance and behavior and parents’ behavior.
Evaluation Design Experimental: Random assignment was used to determine which applicants were accepted at 3 of BELL’s 5 sites in 2005: 2 sites in Boston and 1 site in New York City. In total, 1,917 youth applied to BELL Summer at these 3 sites, more than double the 750 slots available (250 at each site). Of these applicants, 1,225 had signed parental permission to be in the study before being informed of whether or not they would be accepted into the program. No race or gender differences were found between those willing and not willing to participate in the study. Those who did not choose to participate in the study were not excluded in the lottery to decide who would be accepted to the program. Some of the youth with parental consent were excluded either for cost considerations (i.e., families with multiple children enrolled in BELL were more cost effective to study) or because randomization was inappropriate based on the number of program slots available at certain sites (i.e., randomization was not feasible because either no slots were available for some applicants, or there were more slots available than potential applicants). This left a total sample size of 1,087 applicants. Of these, complete youth test and survey data were available for 835 applicants for an overall weighted response rate of 78% for the control group and 78% for the program group. Parent survey response rates were somewhat lower at 73% and 70% for the control and program groups respectively. No significant differences were found between the program and control groups based on observed characteristics.
Data Collection Methods Secondary Source/Data Review: Data on race, gender, and participation status of each applicant were obtained from the application forms the parents filled out and augmented with information from BELL staff on program participants.

Surveys/Questionnaires: The parent surveys covered the types of learning activities the child engaged in during the summer of 2004 (briefly) and 2005 (in great detail), the child’s behaviors, parent involvement in youth’s educational development and other activities, background data, and parent activities. The youth surveys focused on the children’s perceptions of their academic skills.

Tests/Assessments: The Gates-MacGinitie reading test, which measures reading skills and covers both vocabulary and comprehension, was administered to all children in the study. Youth entering Grades 3 to 7 were also administered the Perception of Ability Scale Score (PASS), which is a self-evaluation instrument appropriate for youth in Grades 3 and higher (Hay et al., 1997), measuring perceived ability in reading, math, and general learning. For those entering Grades 1 and 2, perceived ability was measured using the Academic Perceptions Inventory (API, Soares & Soares, 2000) in reading and arithmetic. The parent survey includes a subset of questions from the Social Skills Rating System (SSRS, Meisels, Atkins-Burnett, & Nicholson, 1996) that measure positive social skills (cooperation, assertion, self-control and responsibility) as well as problem behaviors such as aggressive acts, poor temper control, and sadness and anxiety.

References:
Hay, I., Ashman, A.F., & van Kraayenoord, C.E. (1997). Investigating the influence of achievement on self-concept using an intra-class design and a comparison of the PASS and SDQ-1 self-concept tests. British Journal of Educational Psychology, 67, 311–321.

Meisels, S. J., Atkins-Burnett, S., & Nicholson, J. (1996). Assessment of social competence, adaptive behaviors, and approaches to learning with young children. Washington, DC: National Center for Education Statistics.

Soares, L. M., & Soares, A. T. (2000). Academic perceptions inventory: Test manual/Advanced level. Trumbull, CT: Castle Consultants.
Data Collection Timeframe Data were collected between August 2005 and January 2006.


Findings:
Formative/Process Findings

Recruitment/Participation During the summer of 2005, BELL participants were less likely to be at home, in summer school, or in another program than nonparticipants (p < .01), though 19% of the control group was in another program and 17% was in summer school. The program group was also less likely to be with their parents and more likely to be with program staff. No differences were found between program and control youth on hours per week spent without someone over the age of 12. This last result also held when evaluators looked specifically at the weekday hours when the BELL program was in operation.


Summative/Outcome Findings

Academic Statistically significant positive impacts on the Gates-MacGinitie reading test scores were found in children who attended BELL Summer (p < .05). When compared to students who attended the same number of school days, BELL Summer scholars had higher reading comprehension and overall reading scores.

On average, parents in the control group reported that their children spent almost 12 hours per week in academic activities and read 6.6 books in July 2005. Being in the program group significantly increased both of these numbers by about 50%; hours per week in academic activities by 6.4 hours per week and books read by 3.9 (p < .01). Some parents of program youth reported that their children read as many as 50 books during the month.

In order to participate in BELL, members of the program group had to reduce their time in other activities. The largest estimated significant impacts were found on time spent watching TV and playing computer games, doing chores, attending camp, attending other academic activities, hanging out with friends, using the Internet/computer, and attending cultural activities. Smaller but still significant reductions were found for participation in other academic programs; sports, music, or arts; volunteer work; and religious activities (all p < .01 except volunteer work and religious activities, which were significant at p < .05). No impacts were found for summer school or caring for other children.

Estimated impacts on the two measures of academic self-concept were insignificant and fairly small.
Family Significant impacts were found for parents encouraging their children to read and actually reading to their children (p < .05).

Estimated program impacts on parents’ own summer education activities were not significant. However, parents of program group members had a 9% higher probability of taking computer classes than parents of control group members.
Youth Development No effects were found for measures of children’s behavior.

© 2016 Presidents and Fellows of Harvard College
Published by Harvard Family Research Project