The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.
A Profile of the Evaluation of Boys & Girls Clubs of America
Profile Information
- Full Text (HTML)
- Request reprint permission
- How to cite
- About the OST Database
Profile Updated
May 9, 2011
Program Description
| Overview | Boys & Girls Clubs of America (BGCA) provides fun, safe places for youth during out-of-school hours, where they can be involved in caring relationships with adults and peers and feel a sense of membership and connectedness. BGCA provides varied and diverse programming supported by caring staff. |
| Start Date | 1906 as the Boys Club Federation of America (changed name to Boys Clubs of America in 1931 and to Boys & Girls Clubs of America in 1990) |
| Scope | national |
| Type | after school, summer/vacation, weekend |
| Location | urban, suburban, rural |
| Setting | community-based organization, public schools, recreation center |
| Participants | kindergarten through high school students |
| Number of Sites/Grantees | approximately 4,300 Clubs nationally |
| Number Served | approximately 4.5 million children annually |
| Components | Based on the assumption that youth benefit from having safe places where they can come as needed, Clubs traditionally do not require youth to arrive at particular times or attend particular activities. BGCA has devised a variety of targeted programs over the years to reflect shifting policy ideas and public opinion about what types of services and experiences youth need to be successful and which youth most need those services and experiences. |
| Funding Level | $230 million (2009) |
| Funding Sources | public grants, corporations, investment transactions, foundations, Club dues, special events, individuals, investment income, and trust funds/miscellaneous |
Evaluation
| Overview | For the 2005 report, 21 evaluations of BGCA programs and activities over 20 years were examined to determine Club membership’s overall impact and the diverse range of developmental experiences to which members are exposed. The evaluations reviewed cover general Club experiences and three types of targeted programs: delinquency prevention, education/technology, and job readiness. The 2009 report provides findings from the national longitudinal study of Club participants, which aimed to test the degree to which teens’ overall experiences in BGCA might prove to be more than the sum of Clubs’ programmatic parts. |
| Evaluator | Public/Private Ventures (P/PV) |
| Evaluations Profiled |
Beyond Safe Havens: A Synthesis of 20 Years of Research on the Boys & Girls Clubs Making Every Day Count: Boys and Girls Clubs’ Role in Promoting Positive Outcomes for Teens |
| Evaluations Planned | P/PV is conducting a longitudinal evaluation to examine Club experiences and their effects on participants. |
| Report Availability |
Roffman, J. G., Pagano, M. E., & Hirsch, B. J. (2001). Youth functioning and experiences in inner-city after-school programs among age, gender, and race groups. Journal of Child and Family Studies, 10(1), 85–100. Anderson-Butcher, D., Newsome, W. S., & Ferrari, T. M. (2003). Participation in Boys and Girls Clubs and relationships to youth outcomes. Journal of Community Psychology, 31(1), 39–53. Arbreton, A., Bradshaw, M., Metz, R., & Sheldon J. (2008). More time for teens: Understanding teen participation—frequency, intensity and duration—in Boys & Girls Clubs. Philadelphia: Public/Private Ventures. Available at: www.ppv.org/ppv/publication.asp?section_id=23&search_id=0&publication_id=231 |
Contacts
| Evaluation | Amy J. A. Arbreton Senior Research Fellow Public/Private Ventures Lake Merritt Plaza 1999 Harrison Street, Suite 1550 Oakland, CA 94612 Tel: 510-273-4600 Email: aarbreton@ppv.org |
|
| Program | Boys & Girls Clubs of America National Headquarters 1275 Peachtree Street NE Atlanta, GA 30309-3506 Tel: 404-487-5700 |
|
| Profile Updated | May 9, 2011 | |
Evaluation 1: Beyond Safe Havens: A Synthesis of 20 Years of Research on the Boys & Girls Clubs
Evaluation Description
| Evaluation Purpose | To (a) review evaluations from over 20 years on discrete BGCA programs and on general Club experiences and to identify areas in which Club participation was associated with positive youth outcomes, (b) describe programs and Club strategies that contribute to successes and challenges, and (c) identify ways that the overall “Club experience” (rather than participation in any one discrete program) may affect participants’ health and well-being. |
| Evaluation Design | Quasi-Experimental and Non-Experimental: Twenty-one BGCA evaluations were included: 7 studies of 4 delinquency prevention programs, 7 studies of 5 education/technology initiatives, 4 studies of 2 job readiness programs, and 3 studies of overall Club experiences. About a third of studies, particularly in the areas of job readiness and education, were conducted fairly early in implementation and focused on how effectively staff met specific program goals (e.g., implementing curricula, recruiting youth, serving youth with specific needs). A few studies used surveys to assess program satisfaction or beliefs that the program was helpful. The remaining two thirds assessed change over time in youth’s beliefs, attitudes, or behaviors, using pretest/posttest assessments. About half of these used a nonrandomly assigned comparison group of other nonprogram Club members or participants in other organizations. Three studies did not examine discrete programs but provided data about youth involved in Clubs, their participation and experiences, and the Club’s role in their lives. When specific programs/studies are not specified in the findings, it can be assumed that the finding summarizes across programs/studies. |
| Data Collection Methods | Document Review: Studies of the following delinquency prevention programs were reviewed: Project SMART/Life Skills = 1, SMART Moves (SM) = 3, Gang Prevention through Targeted Outreach (GPTO)/Gang Intervention through Targeted Outreach (GITTO) = 2, and Teen Initiatives (TI) = 2. Studies of the following education/technology programs were reviewed: Project Learn (PL) = 3, Educational Enhancement Program (EEP) = 1, Project Connect (PC) = 1, Operation Connect (OC) = 1, and NetSmartz = 1. Studies of the following job readiness programs were reviewed: Broader Horizons (BH) = 1 and Career Prep (CP) = 3. Studies of the following overall Club experiences were reviewed: interviews with a random national sample of male Club alumni in 1985, interviews with a second random national sample of male and female alumni in 1999, and a 1996 study with survey and attendance data for youth ages 10–18 at 5 large urban Clubs. |
| Data Collection Timeframe | Evaluations included were published from 1985–2004. |
Findings:
Formative/Process Findings
| Activity Implementation | In the three Project Learn studies, homework help was the most consistently implemented component. Most Clubs also implemented an incentive piece (e.g., field trips, computer time). Interviews and usage reports data found that, at the time of the study, 13 of the 14 pilot Operation Connect Clubs across the country were operational, technology centers were open 30–40 hours per week, and technology access was provided equally to girls and boys and for all age groups. According to overall Club studies, the variety of activities offered was associated with youth’s positive responses to how interesting they found activities, the level of support they received, and their experience of leadership opportunities. When asked about their most memorable activities in the 1985 Club study, the most frequent responses were organized team sports (79%); recreation, exercise, and games (63%); swimming lessons (30%); and arts, crafts, and shop (26%). According to the 1996 Club study, programs included both structured activities across Clubs’ core areas along with time for more informal interactions with staff and peers; 80–100% of youth reported spending time “hanging out with friends” during the program over the previous 4 weeks. Of youth surveyed in the 1996 Club study, just over three quarters reported having various opportunities to participate in leadership roles at the Club over the previous year, and just under a quarter participated in community service activities during that period. |
| Parent/Community Involvement | The presence of technology in Operation Connect was reported to increase local community members’ positive perception of Clubs as providing activities beyond sports and recreation. Clubs’ partnerships with agencies such as schools, probation and police officers, and other community-based providers provided benefits such as facilitating referrals and access to school academic records. However, Clubs struggled to build and maintain partnerships. Clubs with strong ties to community agencies were able to expand youth opportunities. For example, one Club offered youth a leadership course led by staff from a local U.S. Army post. At another Club, a local advertising agency offered a course in which youth designed and produced an antigang ad campaign. Broader Horizons and Career Prep also relied on the experience and expertise of professionals in the community to host field trips, describe their jobs, or in a few CP programs, lead training sessions. Strong community partnerships also enabled CP staff to more easily find employment for program graduates. Career Prep staff found that many employers were reluctant to hire high-risk youth served by CP. To help employers understand CP’s value and the qualities youth could bring, CP staff called employers to set up appointments to do program presentations. Several delinquency prevention program staff also presented to partner agencies about the Club’s mission and programs. Since partner agencies often had high staff turnover, presentations were most successful when done frequently and regularly. Given limited time available to both Club and partner agency staff, some Clubs found that it was most useful to focus on developing relationships with just a few key agencies. Staff at one gang prevention program indicated that they initially reached out to several agencies, with little success. In year 2, they focused on fewer agencies and developed a strong partnership with at least one. Devoting staff time specifically to partnerships was often successful. For example, some sites received extra funds to partner with other agencies, and these sites had the most partnering successes. In contrast, most delinquency prevention Clubs didn’t have a staff member specifically responsible for community relationships, and it was difficult for staff to manage such relationships on top of other responsibilities and time constraints. Some Clubs sought ways to provide mutual benefits to partners in order to encourage collaborations. For example, one Club’s partnership with a probation officer allowed the officer to use the Club’s space to host an alternative high school during the day. Partnerships tended to last longer when they involved ongoing, regular collaboration. For example, some Teen Initiatives and Career Prep sites partnered with probation departments so that teens could fulfill community-service requirements through Club activities. |
| Program Context/ Infrastructure | Of youth surveyed in the 1996 Club study, 63% reported that Clubs provided safe places, 60% reported that activities were new and different from experiences they had had before and provided interesting opportunities, 59% felt a sense of belonging to the Club (that they mattered and were listened to), and 59% reported involvement in Club decision-making, by helping establish rules or making choices about/providing input on activities. Within Operation Connect’s 1st pilot year, technology use was integrated into Clubs’ other educational programs. Over time, Educational Enhancement Program activities became increasingly integrated into the overall BGCA program. For example, EEP leaders began to use other Club activities to integrate learning opportunities, such as in cooking, where youth could practice reading and writing recipes. |
| Program–School Linkages | Project Learn studies suggested that PL helped to build or solidify relationships with schools and teachers because it gave teachers a concrete idea of how Clubs could help youth (e.g., through homework help). In Project Learn sites located in schools, teachers assumed the Club was mostly a childcare and recreation organization and, as a result, were hesitant to share space and resources. In cases where Club staff clarified PL’s purpose, particularly its educational components, relationships with schools were stronger. In order to facilitate stronger program–school relationships, one delinquency prevention program hired teachers as staff. |
| Recruitment/ Participation | In studies of targeted outreach for delinquency prevention programs, all Clubs came close to or exceeded their outreach/recruitment goals. For example, GPTTO sites met about 88% of their target recruitment of 50 new youth over a 9-month period, and GITTO sites met about 97% of target numbers. Teen Initiatives met and often exceeded their target of recruiting 50 new teens per year. Youth recruits in delinquency prevention programs were often at-risk and therefore unlikely to have participated without outreach efforts. For example, 60–80% of recruited youth in Teen Initiatives received free or reduced-price lunches or were living in public housing. In addition, the GPTTO/GITTO study found that 64% of GPTTO youth and 96% of GITTO youth exhibited multiple personal or social characteristics that would put them at risk for gang involvement, including having family members and friends who belonged to gangs, living in a neighborhood with many gangs, or having a record of delinquency. Delinquency prevention programs demonstrated sustained involvement for many recruits. Teen Initiatives retained about 50% of recruits 2 years into the study; 1 year after joining, approximately 70% of youth were still attending the Club at least once a month. Schools were the primary referral source for Career Prep and several GPTTO and Teen Initiatives programs. Teen Initiatives staff used schools to post fliers and talk to counselors as a way to recruit new youth. Staff and parents reported that the increase in technology due to Operation Connect made the Club a more “fun” place to be. In Clubs with strong technology centers, more youth participated. Staff also felt there was increased Club attendance due to new labs. Career Prep enrolled nearly 14,000 youth over 4 years, ranging from an average enrollment per Club of 137 in year 1 to 60 in year 4. Between two thirds (in year 1) and three quarters (in year 4) completed CP’s pre-employment training. CP served many minorities (in years 3 and 4, 69% were African American or Hispanic) and youth with targeted risk characteristics, particularly in earlier years when risk status was stressed as a criterion for involvement (e.g., 27% of participants were either out of school or attending alternative schools, and 18% were juvenile offenders in year 2). Several programs defined recruitment, particularly of high-risk youth, as a main goal. Programs found three recruitment strategies to be most effective. First, street outreach worked well for delinquency prevention programs. In Teen Initiatives and some GPTTO sites, direct youth outreach was the primary recruitment method; TI staff noted that this was their most successful strategy, partly because it allowed them to reach teens in nontraditional places, like game rooms and housing developments, and to talk to teens “on their level.” Several Clubs reported that outreach was most successful when they hired community members with whom families and youth felt comfortable. Second, school and family referrals were an important outreach source for delinquency prevention programs. Third, partnerships with community agencies, especially schools and juvenile justice agencies, were key to recruitment for TI and Career Prep, both of which targeted high-risk and hard-to-reach teens. To ensure that youth stayed involved, some Clubs tailored programs to youth’s interests, needs, and schedules. In efforts to retain youth, Career Prep offered the program at different times of the day and week and at different intensity levels so that youth could come for long meetings over a shorter period of time or for short meetings over a longer period of time. Some of the issues high-risk youth brought (e.g., behavioral problems, emotional and personal needs, poor academic and learning skills, negative attitudes about the traditional job market) compelled CP staff to cover relevant topics (e.g., learning skills) in workshops and, in the 2nd year, to incorporate more career exploration, goal-setting activities, and educational guidance. The Teen Initiatives and gang prevention studies also indicated that programs’ appeal to youth was based on Clubs’ ability to create and adapt programming to meet their needs and interests. Some programs worked to create fun activities to hold youth’s attention and keep them coming back. For example, youth had difficulty understanding and using some of the materials in the job preparation programs, and staff felt that programs could benefit from integrating more fun into activities. This was especially true in Career Prep, in which high-risk youth were turned off by CP’s school-like atmosphere and needed more hands-on activities to keep them engaged. To meet this challenge, staff continued to develop interesting activities, games, and role-plays over CP’s 4 years. Providing rewards and incentives was a useful strategy for retaining youth who were more difficult to involve in Club activities. Broader Horizons gave participants T-shirts and held awards ceremonies. Some Career Prep staff used cash incentives to keep youth involved. In some programs (e.g., Career Prep and some Teen Initiatives Clubs), adjudicated youth were required to continue in the program as part of their sentence, which helped with retention. The 1985 alumni report indicated that most respondents joined Clubs around the age of 10 and stopped going at about age 14 (54% left by age 14; 67% by age 15). The average length of membership was 4.3 years. The 1999 alumni report revealed the same average age for joining but indicated a later age for leaving (15.4 vs. 14.3). In Club studies, recognizing and involving youth was seen as a way to engage and retain youth and promote a sense of belonging to Clubs. Staff reported routinely praising youth, providing a “youth of the month” award to recognize Club members, and decorating spaces with artwork reflective of the community served. Many Clubs designated space that youth could call their own, such as youth lounges, in which they could hang out with friends. In the 1996 Club study, positive adult–youth relationships were rated as one of the most important reasons why youth came to Clubs. For the 1999 alumni study, over three quarters of alumni recalled participating in recreation, organized sports, and social activities, one quarter recalled involvement in career development, and one third recalled involvement in tutoring/homework help. According to attendance data gathered over 4 weeks in the 1996 Club study, youth typically came to Clubs 2–3 times a week, stayed almost 3 hours, and were involved in a range of activities, including sports, social recreation, education, arts, leadership, and service. The most popular types of activities were sports/physical education and social recreation (with half to three quarters of youth participating), followed by education and skill-building (drawing 17% to one third of youth). Leadership and community service reached 5–10% of youth. In the 1996 Club study, majorities of Club members (between 60% and 95% across the 5 Clubs) reported that the Club was their only after school activity. |
| Staffing/Training | Sufficient time for staff to work one-on-one with youth proved to be essential to program success. In Broader Horizons, several youth needed individualized attention to ensure that they understood and completed the exercises. In Career Prep, the increased focus on high-risk youth in year 2 went hand-in-hand with more individualized attention to youth’s needs; case management was made an integral part of CP after year 1, which enabled staff to spend more one-on-one time with youth, focus on skill areas needing improvement, and get to know youth in ways that encouraged continued participation. In GPTTO/GITTO, Project Learn, and Educational Enhancement Program, staff regularly interacted with and questioned youth about school progress, grades, and educational activities. The Teen Initiatives study also highlighted the importance of staff’s connections to youth, high expectations, and awareness of teens’ interest, needs, and goals. Hiring additional program staff was a strategy Clubs used to ensure that all youth received individualized attention and would therefore perceive support from staff. For example, Teen Initiatives Clubs reported hiring staff devoted specifically to serving teens; more than 80% of youth reported that at least one staff member served as a confidant and source of support, and about 75% reported that at least one staff member knew their interests and goals. In the GPTTO study, almost all youth reported that they received support and guidance from at least one staff member; more than half reported that they received support from two or more staff. Staff turnover was particularly troublesome for many programs, as high turnover often diminished their capacity to provide quality programming and forge lasting staff relationships with youth, other staff, and outside agencies. For instance, Teen Initiatives programs had extensive turnover in key staff positions, and TI youth reported low levels of adult support. Recruiting skilled staff for specific programs, such as technology or case management, was of particular concern at some programs. For example, implementation of academic components in Career Prep, Broader Horizons, Educational Enhancement Program, and Project Learn meant that Clubs had to require a higher level of skills in staff. Similarly, technology programs found that having tech-savvy staff working with youth was key to success. Promoting staff from within Clubs to other positions in new Club programs was a strategy used to ensure high quality and experienced staff. In the later years of Career Prep, staff were more likely to be hired from within and to have experience working with high-risk youth. Programs found that ensuring staff buy-in to new programming from the beginning was crucial to retaining staff. For instance, Clubs experienced initial staff turnover during Project Learn’s implementation, as some staff resisted the shift in Club focus and philosophy. Upon hiring new staff, Clubs discussed the goals and intentions of the educational strategy and sought buy-in from prospective staff. Clubs found that it was important to provide sufficient staff training to meet the demands of programming changes, particularly when implementing programs that involved high-level skills or experiences, such as the educational and technology programs. For example, Broader Horizons staff wanted to provide youth with better information on different kinds of jobs and their requirements and therefore needed specific training to obtain this knowledge. In the 1996 Club study, 73% of Club members indicated that there was at least one adult at the Club who provided positive support in their lives. In alumni studies, 37% of those surveyed in 1985 and 48% in 1999 recalled a particular staff member “as having been important to their life.” The alumni often said that this staff member provided a role model or father figure for them, helped them learn skills (like sports), taught them right from wrong, and provided emotional support. |
| Systemic Infrastructure | In the final Operation Connect report, findings suggested that Clubs’ technology centers provided computer and Internet access for youth who did not have access at home. For 1985 alumni study respondents, almost three quarters felt that there were not very many (59%) or no (17%) other organized activities available to them during after school hours when they were involved in Clubs. |
Summative/Outcome Findings
| Academic | Educational Enhancement Program youth reported greater engagement and enjoyment in reading, verbal skills, writing, and tutoring; and enjoyment of geography than non-EEP youth. Reading, spelling, history, science, and social studies test scores; grades; and school attendance also favored EEP youth over non-EEP youth. In the Educational Enhancement Program study, teacher reports indicated more positive reading, writing, and games skills; overall school performance; and interest in class material for Club members (both those in EEP and those not) than comparison youth (those in other organizations). Club youth also had better math grades than comparison youth. One of the Project Learn evaluations using a pretest/posttest survey design indicated that a slightly higher percentage of youth finished their homework, and that discussions about books increased. Broader Horizons youth interviewees reported that their attitudes toward school improved due to BH. In one of the alumni studies, 59% of alumni respondents felt that the Clubs had helped in their school life. In addition, alumni who reported greater familiarity with staff were more likely to report positive effects of Club attendance on school life. When asked how the program had changed their attitudes toward school, some Broader Horizons youth interviewees were interested in going to college, but others realized that there were also possibilities open to them if they decided not to continue on to college. One study found that relative to comparison youth, GPTTO youth were more likely to seek an adult for homework help and GITTO youth spent more time on homework. |
| Family | In one of the alumni studies, 60% of alumni respondents felt the Clubs helped in their family life. Alumni who reported greater familiarity with staff were more likely to report positive effects of club attendance on family life. One study found that GITTO youth improved their family relationships. |
| Prevention | Two studies found that SMART Moves participants showed more awareness of the risks of drug use than comparison youth. One of these studies further found that, over time, youth in SM maintained or increased their ability to refuse alcohol, marijuana, or cigarettes, while comparison youth showed a decreased ability to refuse them. A study of an SM booster program found little evidence of additional impacts on participating youth, compared with youth who participated only in the basic SM program. Delinquency prevention program studies suggested that participants engaged in fewer delinquent behaviors. For example, in one SMART Moves study, participants reported less alcohol, marijuana, and cigarette use than comparison youth. Another study found that public housing areas with a Club that offered SM had a lower presence of crack cocaine and lower rates of drug activity than areas without SM. A GPTTO/GITTO study found that relative to comparison youth, GPTTO youth showed decreased marijuana use and were less likely to cut class. GITTO youth showed a decrease in cutting class and skipping school. More frequent participation in GPTTO/GITTO was associated with more positive outcomes, such as fewer gang-related behaviors and less contact with the justice system. Some delinquency prevention efforts showed little to no effects on certain delinquency outcomes. For example, one SMART Moves study found no difference between participants and comparison youth in alcohol, cigarette, or other drug use, and another found no differences in alcohol or marijuana use. A GPTTO/GITTO did not find any differences in participants’ likelihood of joining or leaving a gang. Of participants in one of the alumni studies, 74% felt that Clubs helped them to be able to avoid difficulty with the law. |
| Workforce Development | The percentage of Operation Connect youth reporting that computer use at Clubs made them think about jobs in technology increased; fewer youth over time reported not wanting to work with computers or thinking that there was little need for computers in most jobs. About half (47%) of surveyed Broader Horizons youth agreed that BH helped them develop their talents and skills and 87% indicated that BH taught them about setting career goals; 45%, however, reported that their career plans did not change due to BH. The Career Prep study found that an average of 45% of participants in year 1 and 53% in year 4 were placed in jobs. Although most employed participants worked 20 or fewer hours per week, a substantial percentage (39% in year 4) worked 31 to 40 hours per week. In year 4, 72% of employed participants received wages above minimum wage. Of respondents in a Club alumni study, 64% felt that Clubs helped them in their work life. |
| Youth Development | One of the Project Learn evaluations using a pretest/posttest survey design indicated that youth discussions with adults increased. For Operation Connect youth, computer literacy scores increased over time, particularly in basic computer operations and file management. Computer use overall also increased over time, with schoolwork, games, and Web searching the most popular activities. About 22% of Club members were frequent users (3 or more hours per week), and these youth generally had more positive outcomes (e.g., greater increases in computer literacy scores) and were more likely to use computers at Clubs to do schoolwork, learn more about computers, or learn about college. Among frequent users, boys outnumbered girls nearly 2 to 1. NetSmartz youth improved Internet safety awareness from pretest to posttest; on a scale of 1–7, youth 12 and under increased from 5.2 to 6.0, while teens increased from 5.7 to 6.3. For younger youth, one area in particular that changed was in awareness that it was not safe to meet in person someone whom they had met online, or to tell them their real name. For teens, awareness increased most in understanding the dangers of meeting someone in person after communicating online, posting a picture online, being rude, or telling someone online where they go to school. The study also showed an increase in the number of youth who reported that they would talk to an adult about Internet safety (e.g., talk to an adult if something on the Internet made them uncomfortable). Similar changes were not found in Operation Connect studies. Almost all interviewed Broader Horizons youth reported that BH helped improve their self-esteem. Staff from two Broader Horizons Clubs felt that BH created cohesive peer groups—youth met other youth through participation and developed friendships that lasted beyond BH. In the 1985 alumni study, when asked about the most important way that Clubs affected their life, the most frequent responses of the men interviewed involved learning how to interact socially (58%). In the 1985 alumni study, 82% of respondents reported that they made good friends among other boys, and 67% reported that the Club helped them learn how to get along with people of other races, religions, and backgrounds. Over half of respondents in the alumni studies reported that being in the Club helped them learn how to be a leader. The majority of respondents in one of the alumni studies felt that the Clubs helped them in their health and fitness (83%) and ability to work well with others (85%). Across Club studies, youth’s Club tenure was an important factor in determining the extent to which they had positive experiences or felt the Club helped them. For example, among alumni interviewed in 1985 who participated for 5 or more years, 60% suggested that relationships they formed with staff were critical in helping them succeed later in life and stay out of trouble, compared with 34% of those who participated 3–4 years and 17% of those participating 1–2 years. Similarly, the 1996 Club study found that youth whose Club tenures were longer reported higher levels of developmental experiences in all seven areas examined by the study (adult support, sense of belonging, sense of safety, opportunities for input and decision-making, leadership, engaging in interesting and challenging activities, and community service). One study found that relative to comparison youth, GPTTO youth increased their involvement in after school activities. |
Evaluation 2: Boys and Girls Clubs’ Role in Promoting Positive Outcomes for Teens
Evaluation Description
| Evaluation Purpose | To address the question: What role do Boys & Girls Clubs play in influencing change in older youth outcomes? |
| Evaluation Design | Quasi-Experimental and Non-Experimental: Ten BGCA Clubs across the country were selected for the study based on a set of criteria that included their prior success in reaching and serving relatively large numbers of older youth, location in an urban setting, and utilization of an electronic enrollment and attendance tracking system. Baseline surveys were collected at the 10 Club sites from 422 seventh and eighth grade Club participants. Follow-up surveys were completed by 322 youth whether or not they were still participating in Clubs (76% of the initial 422 youth whom completed the baseline survey) approximately 30 months after the baseline survey, when participants were in ninth and tenth grade. In addition, interviews were conducted with 56 ninth grade youth (selected from surveyed youth) and 86 program staff who worked directly with older youth across the 10 programs. Interviews were conducted both with staff who work with all ages of youth (e.g., Club and program directors) as well as staff who work solely or primarily with older youth (e.g., teen directors or teen advocates). |
| Data Collection Methods |
Interviews/Focus Groups: Club staff interviews were conducted to understand staff’s perspectives on the teen programming provided at the Clubs. Interviews with ninth-grade youth sought to gain a richer and more nuanced understanding of teens’ perceptions of how the Club fit into their lives, what older youth do when they are at the Club, and what they gained and believe they had to gain from participating in its programs. Secondary Source/Data Review: Club enrollment information on the selected youth and daily Club attendance data were gathered from the Clubs. Surveys/Questionnaires: Youth surveys were conducted to understand how older youth use their out-of-school time, what activities they participated in outside of the school day both in and out of the Club, their experiences in the Club, and how their attitudes and behaviors change as a result of Club participation. |
| Data Collection Timeframe | Data were collected between 2006 and 2008. |
Findings:
Formative/Process Findings
| Activity Implementation | The Clubs engaged older youth by offering a broad array of programming, along with a time and a place to hang out with friends, other positive peers, and supportive adults. All of the Clubs had a separate space for older youth, either in the form of a teen-only room or a larger teen-only center. In addition, the Clubs were open long hours after school and over the summer months, allowing older youth to attend at the times that were best for them. Among youth who had been to the Club in the past four months, over half (56%) reported participating in at least four of the following types of activities: informal activities, athletic programs, academic programs, other formal activities like classes or prevention programs, and paid or volunteer work. Among their activities in the Clubs, about three quarters of youth reported using technology such as computers, video, or digital music for fun (74%) and/or for schoolwork (71%); about two thirds reported playing sports (68%); 46% reported doing arts and crafts activities; over a third reported participating in specific leadership programming (36%); and a third attended a class in drama, dance, or performing arts (33%). Of those who had been to the Club in the preceding year, 82% reported participating in at least one leadership role at the Club (such as serving on a youth council or leading an activity) during that time, and half participated in three or more such roles. |
| Program Context/ Infrastructure | Among youth who completed the final survey, 79% rated the level of safety at the Club as 8 or higher on a scale of 1 to 10, where 10 is the highest level of safety. By comparison, only 54% of youth rated the safety of their school as favorably, and only 38% gave a similarly high rating for the neighborhood surrounding the Club. Youth described the Clubs as providing an important space for them where they could comfortably hang out and stay away from unhealthy choices and negative peer pressure. Almost all of the youth surveyed said there was at least one supportive peer at the Club (93%), and that the Club was a place where there was peer cooperation (95%). The youth also reported that the Club was a place where they had opportunities to develop skills (90%), had fun (91%), and felt a sense of belonging (91%). |
| Recruitment/ Participation | Over the 30 months of the study, over half of the youth (52%) attended more than 122 days (roughly once a week), of which almost a third (30%) attended 244 or more days (roughly 2 –3 times per week) and 13% attended 366 or more days (roughly 3–4 days per week). The frequency of attendance was lower for youth when they reached the ninth and tenth grades: these youth attended about one day per week on average, compared with about two days per week when they were younger. In interviews, many youth reported that they combined their Club activities with other pastimes during out-of-school hours. A common theme described by participants in interviews was the feeling that, even if they were not using the Club right now, they could go back at any time. Older youth were likely to participate with greater frequency during the transition from middle to high school if they were fully engaged in Clubs through a variety of activities, had friends who came to the Club, were involved in leadership roles at the Club, and had first become involved in the Clubs as preteens or even younger (p < .05 for all). Relative to those who had been to a Club in the past four months, a significantly higher percentage (p < .05) of those who had not attended reported on the final survey that they were spending time caring for siblings or working for pay. Of those who said they did not plan to go back to the Clubs, the most common reasons reported on surveys were that they had too much else to do (29%), moved (18%), got a job (13%), or were not interested in the activities (13%). |
| Staffing/Training | Almost all of the Clubs had at least one staff person whose time was devoted specifically to working with older youth, and all of the Clubs had several staff who interacted with older youth on a regular basis. Almost all older youth participants said there was at least one supportive adult at the Club (96%) and that at least one adult at the Club set high expectations for them (96%). Staff and youth described the Club as a place where staff built relationships, and said that those individual relationships created the bridge to programming; conversations about character, school, and healthy lifestyles; and opportunities for staff to serve as role models. In interviews, Club staff described how they supported youth’s positive development by paying attention to youth, being there to support them, and providing both formal and informal opportunities for activities and interactions with peers and other staff. In interviews, youth described the importance of their relationships with staff for their learning and positive development at the Club. Both staff and youth reported in interviews that they felt staff were accessible and approachable. |
Summative/Outcome Findings
| Academic | Youth who came to Clubs with greater frequency were significantly more likely than less frequent attendees to report decreases in skipping school (p < .05), increased academic confidence (p < .05), and increased school effort (p < .10) on the follow-up surveys. No significant relationships were found between frequency of participation and the other academic success outcomes (liking school, importance of school, teacher connectedness, report card grades, or school suspensions). More frequent Club participation was associated with increased academic confidence and school effort after a minimum of 52 days of participation (or 1 day every other week). More frequent Club participation was associated with decreases in skipping school after a minimum of 244 days (or 2 days per week). |
| Prevention | Youth who came to Clubs more frequently reported decreased numbers of negative peers as friends (p < .10), decreased numbers of times stopped by the police (p < .05), and decreased levels of aggression (p < .10) on the follow-up survey. More frequent attendees also reported a lower likelihood of starting to carry a weapon (p < .05), smoke cigarettes (p < .05) or marijuana (p < .10), drink alcohol (p < .05), or have sexual intercourse (p < .05). No significant relationships were found between frequency of participation and the other healthy lifestyles outcomes (numbers of positive peers, days of vigorous exercise, arrests, probation, initiation of fights, and initiation of other drug use). More frequent Club participation was associated with decreases in smoking marijuana after a minimum of 52 days of participation (or 1 day every other week). More frequent Club participation was associated with decreases in carrying weapons, being stopped by the police, smoking cigarettes, levels of aggression, and drinking alcohol after a minimum of 122 days (or 1 day per week) More frequent Club participation was associated with lower likelihoods of starting to have sexual intercourse after a minimum of 244 days (or 2 days per week). |
| Youth Development | Youth who came to Clubs more frequently reported increased levels of future connectedness (how much they think about their future and how their current activities help them prepare for the future) on the survey (p < .05). Youth who came to the Clubs with greater frequency were significantly more likely than less frequent attendees to report higher levels of community service involvement (p < .001), increased levels of integrity (knowing right from wrong, p < .10), and decreased levels of shyness (p < .05). No significant relationships were found between frequency of participation and the other character and citizenship outcomes (fairness, open-mindedness, social competence, and problem solving/conflict resolution). More frequent Club participation was associated with increases in future connectedness, community service, and integrity after a minimum of 52 days of participation (or 1 day every other week). |