You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.

www.HFRP.org

The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.

Terms of Use ▼


Program Description

Overview The San Francisco Beacon Initiative (SFBI) was founded by a broad-based group of San Francisco leaders who wanted to transform public schools in low-income neighborhoods into youth and family centers that would serve as beacons of activity uniting communities. Stakeholder groups, facilitated by the Community Network for Youth Development (CNYD) and the Institute for Research and Reform in Education (IRRE), developed a theory of change to be used throughout the initiative to guide its action and management. At the site level, the theory of change emphasized the following four critical developmental supports and opportunities for youth development: supportive relationships, safe places to spend leisure time, interesting and challenging learning experiences, and opportunities for meaningful roles and responsibilities. The initiative aims to help youth through participation in Beacon activities during out-of-school time to develop competencies that will help them become responsible adults.
Start Date Founded in 1994, the first five centers opened between 1996 and 1998. Three more were added in 1999.
Scope local
Type after school, summer/vacation, before school, weekend, comprehensive services
Location urban
Setting public schools
Participants elementary through high school students
Number of Sites/Grantees eight centers
Number Served 7,500 youth and adults from July 1, 1999 through June 30, 2000
Components The centers are designed to be neighborhood gathering places that provide an array of developmental opportunities for community youth in nonschool hours, as well as activities for adults. Youth programs focus on five areas: leadership, career development, arts and recreation, health, and education. As a result of increased national and local attention on school accountability and student performance in recent years, the youth programming has shifted from a broad youth development agenda, equally incorporating these five areas, to one with a strong focus on educational programming. By 2001, all centers were running educational support programs at least 4 days per week. Neighborhood adults can participate in English and computer lessons, community events, and parent support groups. Although some Beacon Centers provide traditional social services, their goals are much broader. As community centers, they are designed to be responsive to the local needs and conditions of specific neighborhoods. The communities’ ethnic makeup, organizational resources, and specific youth and adult needs shape the centers’ operations and offerings. The centers provide activities in the hours immediately after school, during some evenings, during lunch, on Saturdays, and in the summer, and they range from daily and weekly programs to single-time events.
Funding Level $4 million in 2000; $8.4 million in 2003–2004. The 2003–2004 figure includes $2.4 million from the Department of Children, Youth and Their Families and an additional $6 million leveraged by the eight centers, both in-kind and cash. It does not include in-kind contributions that have been provided by the school district (space and janitorial services).
Funding Sources Department of Children, Youth and Their Families (DCYF), Juvenile Probation Department, local private foundations (represented by the Evelyn and Walter Haas Jr. Fund), the school district, California’s Safe Neighborhood and After-School Partnerships Act, the federal government’s 21st Century Community Learning Centers initiative
Other A complex organizational structure links the Beacon Centers together. Each center’s day-to-day operations are overseen by a community-based organization, while a citywide steering committee of funders oversees the entire initiative. The steering committee sets policies and expectations for center operations, provides core funding, and raises additional money for the centers. As the initiative’s operational manager during the first 5 years of the centers’ operations, CNYD helped identify key challenges and brought them to the steering committee’s attention. In addition, CNYD provided centers with technical assistance and training, kept lines of communication open among the partners, brought potential funders and activity providers to the centers, and managed a public support campaign to introduce elected officials and community members to the programs. Through CYND, all center directors met monthly to share information about the institutions and people who provide activities at their centers. Many of those functions are now handled by the executive director of the San Francisco Beacon Initiative and her staff, located in DCYF.

Evaluation

Overview Designed around SFBI’s theory of change, the evaluation aimed to (a) assess the initiative’s progress in achieving its intended outcomes, and (b) analyze how the initiative set out to achieve these outcomes.
Evaluator Karen E. Walker and Amy J. A. Arbreton, Public/Private Ventures
Evaluations Profiled Working Together to Build Beacon Centers in San Francisco: Evaluation Findings From 1998–2000

After-School Pursuits: An Examination of Outcomes in the San Francisco Beacon Initiative
Evaluations Planned none
Report Availability Walker, K. E., & Arbreton, A. J.A. (2001). Working together to build Beacon Centers in San Francisco: Evaluation findings from 1998–2000. Philadelphia: Public/Private Ventures.

Walker, K. E., & Arbreton, A. J. A. (2004). After-school pursuits: An examination of outcomes in the San Francisco Beacon Initiative. Philadelphia: Public/Private Ventures.

Available at www.ppv.org/ppv/publications/publications.asp?search_id=8.

Contacts

Evaluation Karen Walker
Vice President, Director of Research
Public/Private Ventures
2000 Market Street, #600
Philadelphia, PA 19103
Tel: 215-557-4412
Fax: 215-557-4469
Email: kwalker@ppv.org
Program Virginia Witt
Executive Director
San Francisco Beacon Initiative
1390 Market Street, Suite 900
San Francisco, CA 94102
Tel: 415-934-4848
Fax: 415-554-8965
Email: vwitt@sfbeacon.org
Profile Updated December 21, 2004

Evaluation 1: Working Together to Build Beacon Centers in San Francisco: Evaluation Findings From 1998–2000



Evaluation Description

Evaluation Purpose To examine the SFBI’s progress in developing an administrative structure and establishing the centers necessary to achieve the initiative’s long-term goal of improving the lives of the city’s youth.
Evaluation Design Quasi-Experimental and Non-Experimental: Data were gathered on the first five Beacon Centers that opened between fall 1996 and fall 1998. Three of the five centers operate in middle schools, one is in a high school, and one is in an elementary school.

The evaluation included all sixth and seventh graders (Beacon Center participants and nonparticipants) at the three middle schools hosting Beacon Centers. Surveys were administered to participant and nonparticipant youth in November 1998 and November 1999. Survey response rates ranged from 51% to 87% at each center for each round. School records were integrated with survey data to compare the academic performance of the Beacon Center youth with those who do not attend the centers.

In addition, site visits were conducted at the five sites to assess the centers’ implementation.
Data Collection Methods Document Review: SFBI documents were reviewed, including training and outreach materials, progress reports, and budgets.

Interviews/Focus Groups: During biannual visits to the centers, approximately 80 staff members and activity providers and 35 stakeholders were interviewed. Stakeholders include public and private funders, steering committee members, CNYD staff, school-district personnel, and administrators from city agencies. In these interviews, evaluators collected data about the centers’ operations, the host schools, Beacon-school relationships, and the visions of key stakeholders and how well those visions were being implemented. Some interviewees provided information about their institutions’ reasons for getting involved in the initiative, along with descriptions of their involvement. Some also acted as key informants in helping understand local concerns and circumstances.

Observation: From summer 1999 through spring 2001, observers examined Beacon Center activities, looking particularly at such key youth development dimensions as adult-youth interactions, peer interactions, and opportunities for decision making that staff members provided. Observations assessed several attributes of the adult-youth relationships in a subset of activities at each center, including adult responsiveness (encouragement and support of youth’s efforts), instrumental support (helping youth understand and succeed at the task at hand during the activity), and the emotional quality of the relationship (e.g., staff and youth appeared to enjoy each other’s company).

Secondary Sources/Data Review: School records were collected from the San Francisco Unified School District (SFUSD) on gender, ethnicity, grade point averages, suspensions, and attendance. In addition, CNYD oversaw the development of a web-based management information system (MIS) that permitted all centers to enter a variety of information including participant enrollment and attendance data; youth demographics; activities information, including schedules, who administers them (the Beacon Center or a collaborating agency), and the core area each encompasses; and individual staff data.

Surveys/Questionnaires: Surveys served to measure how the students spend their out-of-school time and to document their developmental opportunities. Youth survey responses provide measures of several different constructs: school engagement, self-efficacy, positive reaction to challenge, meaningful roles and responsibilities, and supportive relationships with adults and peers.

Tests/Assessments: Standardized test scores (math and reading Normal Curve Equivalent scores) were collected from SFUSD.
Data Collection Timeframe Data were collected between fall 1998 and summer 2000.


Findings:
Formative/Process Findings

Activity Implementation In fall 1999, each of the eight centers provided 14 to 24 activities for youth.

The centers provided a diverse mix of activities, although more activities focused primarily on two of the five core areas: (a) education and (b) arts and recreation.

The centers provided a broad range of educational enrichment activities, from homework help for individuals to book clubs and structured tutoring and reading programs.

About 75% of the middle school participants surveyed thought the centers’ programming offered them variety and choice.

The most popular adult activities were English as a second language, dance, and computer courses.

Seven of the activities observed were given ratings lower than the midpoint of the adult-youth relationships scale. The activities with the lowest ratings of adult responsiveness and instrumental and emotional support tended to be those that were more instructional in nature (e.g., chess club). In contrast, activities with the highest ratings were mostly leadership activities with high levels of youth planning and involvement (e.g., youth councils).

Four of the five Beacon Centers established safety and support teams to promote a sense of safety. The site without a formal team was a small school with restricted access through a central courtyard; Beacon staff did not think a team was necessary there.
Costs/Revenues Public and private funders adopted unified guidelines for Beacon funding proposals, and core monies were pooled and distributed through joint decisions.

Most of the public funds for the centers’ core operations come from DCYF. Efforts to diversify the public funds available for core support included bringing other city institutions into the initiative and calling for additional state legislation for youth services. For example, initiative leaders helped garner public support for legislation to increase the city’s set-aside of 2.5 cents to 3 cents per $100 of assessed real property value for children and youth services. A portion of those funds provided 85% of the centers’ core operating budget of $350,000 per year. In fall 2000, this legislation passed, increasing the set-aside to 3 cents and extended the program for 15 years.

Seventeen private foundations provided capacity-building grants to centers and 15% of their core operating budgets. They also financed CNYD, the public support campaign, and the evaluation. Most of the local foundations that contributed pooled their resources, giving the steering committee flexibility in deciding how to spend the funds.

In fiscal year 2000, each center received an additional $178,500 to $770,000 from a variety of sources-including the Juvenile Probation Department and the school district, both of which received state and federal funds for after school activities. Local agencies provided in-kind resources by operating activities at the centers and providing space.
Parent/Community Involvement Each center worked with 7 to 14 community partners. The proportion of youth served by the partners varied widely-at one center, partners served 17% of the youth, at another, 90%.

Staff from partnering organizations provided activities in all five core areas, and volunteers and AmeriCorps members helped increase the centers’ abilities to provide services. In addition, some partner organizations provided services such as immunizations for youth and transportation for youth with special needs.

The availability of suitable partners depended largely on the range and number of organizations within the neighborhood. To offset the dearth of services evident in two of these communities, center staff called on agencies in other areas of the city and on community residents, who acted as independent contractors, to provide activities for youth.

Staff who coordinated activities at the Beacon Centers used a variety of strategies to ensure communication among the partners, such as weekly or bimonthly meetings.

The centers had mixed success in fostering community engagement among residents. They were most successful in attracting adults and youth to program activities and in attracting partner agencies from the community to implement programs at or for the centers.

The centers employed neighborhood adults as safety and support team staff and as activity providers. The Meadow Beacon Center, in particular, partially compensated for its surrounding community’s low number of youth-service agencies by hiring community residents as providers.

A public support campaign, managed by CNYD, succeeded in garnering support from local officials, who attended events at the centers and voiced their support.
Program Context/Infrastructure The centers did not provide transportation, and about a quarter of the youth experienced difficulties in getting home after programs ended. In at least one school, where 40% of the youth were bused, a lunchtime activity was offered to counter the transportation problems.

Because elementary and middle schools are designed for use during the day, the centers’ lighting was often inadequate for evening activities. Centers attempted to use portable lights or to persuade the schools to improve lighting, but these efforts had limited success. The question of funding for additional lighting emerged as a serious issue.

Eighty-seven percent of participating middle school youth considered the centers safe.
Program-School Linkages The initiative’s long-term vision included increasing Beacon-school integration, but a consensus on what this means had not been reached. At all centers, Beacon and school staff discussed space issues. At three centers, the cooperation went further, with center and school staff discussing specific students and programs.

Development of a standard agreement about the use and maintenance of school space proved elusive.
Recruitment/Participation The centers met or exceeded their goal of attracting 500 to 1,000 youth and adults at each center annually, whose ethnic diversity mirrored the community’s population. From July 1, 1999 through June 30, 2000, the five centers in the study each served 640 to 1,640 participants—3,746 youth and 1,435 adults in total.

In fiscal year 2000, the centers served 150 to 764 adults, about 25% of each center’s total participation.

The average number of youth and adults served on a daily basis ranged from 51 to 174 across centers. The two centers that served the most people daily (and met the expectation to serve at least 150 people a day) were located in the largest schools.

The proportion of Latino and White youth at the centers reflected the racial makeup of the host schools. Fewer Asians and more African Americans used the centers than the schools’ racial breakdown would have indicated.

In the three middle schools studied, approximately half of youth participants were middle school students, and the rest were evenly split between elementary and high school students. In the high school, approximately 80% of youth participants were high school students. In the elementary school, approximately 70% of youth participants were elementary school students.

The centers served approximately equal numbers of girls and boys.

The Beacon Centers at the three middle schools recruited proportionately more students at risk of academic failure than attended the host schools. Specifically, Beacon youth had significantly lower grade point averages (p < .001 for all three middle school centers), and standardized test scores in math (p < .001 for Valley and Summit, and p < .05 for Meadow). Two of the centers served proportionately more youth who had been suspended from school and (perhaps as a result) fewer youth who had attended all days of school. Additionally, at two schools, Beacon youth tended to have lower reading test scores than non-Beacon youth.

In October 1999, 51% to 79% of the youth at the centers attended activities once a week or less. The centers subsequently added programs that meet more frequently.

Of Beacon youth at each center, 60% to 84% participated in only one activity over a 4-month period.

Beacon Centers relied on several tactics to ensure that their programs were well used by youth in the host schools and surrounding neighborhoods and adults in the community, including placing signs and banners advertising the presence of the centers in highly visible locations both in and outside the schools; printing flyers in several languages, including Russian, Spanish, Cantonese, and Tagalog that list their offerings by season; and advertising in several languages on the radio and in community newspapers.

Beacon staff undertook extensive outreach and recruitment efforts. They attended school staff meetings, and school staff helped to recruit youth to the centers. In the Valley Beacon Center, staff handed out program information in the cafeteria during lunch. At the high school that hosts the Ocean Beacon Center, Beacon staff made announcements over the public address system. In the Eastern Beacon Center, staff maintained a database with all the students’ names and addresses and mailed out flyers as new programs were developed.

Beacon staff felt that word of mouth and one-on-one conversations with youth worked best to advertise the centers and their programs; both they and school staff referred youth to the centers. In addition, Beacon Center safety and support staff said they approached youth hanging out after school to persuade them to participate in Beacon programs.

Initially, identifying who was responsible for youth recruitment and outreach proved to be a program challenge. Providers from subcontracted agencies believed that this responsibility fell to Beacon staff, while Beacon staff thought the responsibility should be shared. As a result of this early confusion, several centers defined more clearly the outreach and recruitment strategies, including who was responsible for these tasks. At the Valley Beacon Center, a staff person was hired specifically to do outreach and educate providers on the best way to advertise their activities.

According to the fall 1999 survey of sixth and seventh graders, the number of host school students who had not heard of their Beacon Center ranged from 7% to 19% at each center.

Beacon Centers’ efforts to make their programs accessible to the communities they served included addressing physical barriers to accessibility; providing all activities, events, and programs for free; offering programs in the early morning, during the day, and in the evening to accommodate youth and parent schedules; and hiring staff who speak languages spoken in the community.

Center after school, summer, and weekend activities were limited to children living outside the neighborhood unless their parents or other adults arranged transportation.

Transportation problems were more critical for elementary school children than for older youth because the younger children could not take public transportation alone, and the elementary schools did not often have “late buses” to transport children who stayed for extracurricular activities.

Centers faced challenges in attracting community adults and youth to leadership roles. At the beginning of the initiative, each community convened a planning group that included agency partners, staff from the lead agencies, school staff, and community residents. Group members helped identify community needs and in some cases laid out a preliminary plan of action. Neighborhood residents (both adults and youth) played a limited role in the original planning councils. As implementation got under way, the councils’ role diminished, and Beacon directors took over the responsibility for convening the councils. Beacon staff continued to work with these councils, but reported that community engagement was low. Councils met irregularly, attendance was sporadic, and Beacon staff were not sure what role they hoped councils would play.
Staffing/Training The centers hired staff members who reflected the neighborhoods’ diversity.

Staff experience varied across and within the Beacon Centers. Beacon directors tended to be the most experienced staff; they typically had several years of administrative experience in other youth-serving organizations. Other staff’s experience ranged widely, from having a few years of experience in other youth-serving organizations (e.g., YMCA), to having relatively recently graduated from college with some experience in youth work, to (in a few cases) having worked as a paraprofessional in a school.

The staff who provided activities were relatively young—many were in their 20s.

Staff from agencies contracted by Beacon Centers to provide youth development activities (e.g., leadership groups and performing arts activities) had backgrounds similar to Beacon staff; they tended to be young and have a few years experience in youth-serving organizations.

Training and technical assistance were available to Beacon staff in several ways. First, CNYD provided specific training at centers. The Eastern Beacon Center, for example, had a small group of disruptive youth and requested that CNYD teach staff how to manage behavioral problems and to provide training in child abuse and conflict management. Center youth-program coordinators also discussed specific job concerns with CNYD staff. In addition, the Bay Area Youth Agency Consortium (BAYAC) supplied centers with AmeriCorps members, who, in addition to working at centers, spent Fridays at BAYAC, where they received training in literacy, youth development, budgeting, time management, CPR and first aid, and meeting facilitation. Lastly, the YMCA provided training to centers in such areas as youth work principles, child-abuse prevention, incorporating leadership into youth programs, and the legal issues involved in working with youth.

Community residents who served as individual subcontractors reported limited access to training. Because training provided by the centers was typically on-the-job, subcontractors, who were not under direct Beacon supervision, did not receive this benefit. Centers did not provide more formal ongoing training opportunities for subcontracted individuals.

Youth-program coordinators focused on and paid attention to the needs of the youth; adult-program coordinators worked with parents and adult community residents to design programs of interest to them; and site managers and Beacon directors responded to the broader needs of school personnel, the community, and partner agencies.

Coordinating Beacon staff became responsible for transmitting youth development principles and effective practices to their providers by dispensing ideas and information in Beacon staff meetings. In addition, the Meadow Beacon Center prepared a provider manual that included information about the youth development framework that the center wanted providers to implement in all activities.

At each center 27% to 56% of Beacon youth did not identify any Beacon adults on whom they could rely for support. When adults at school were included (teachers, counselors, coaches, or other non-Beacon Center adults), the proportion was much smaller, ranging from 10% to 14%.
Systemic Infrastructure Stakeholders made an early commitment to identify the initiative’s common mission, goals, and outcomes. These elements were structured into a theory of change, which identified the strategies and partners responsible for carrying them out. The theory of change has been used throughout the initiative to guide its action and management.

Public and private funders adopted unified guidelines for progress reports, greatly reducing paperwork at the centers.

Establishing standards to ensure high quality programs was difficult; enforcing standards across multiple organizations with different cultures and experience levels added to the challenge.


Summative/Outcome Findings

Youth Development Beacon youth surveyed reported significantly greater opportunities to assume a range of formal, informal, and representation-type leadership roles than did nonparticipant youth surveyed (Valley: p < .01, Meadow: p < .05, Summit: p < .001).

Middle school participants reported spending approximately 2.5 hours more per week in productive leisure activities—art, music, dance, drama, and tutoring—than youth who attended the schools but not the centers. There were significant differences (p < .05 for each) in watching TV (less time spent by participants than nonparticipants), organized sports (more time spent by participants than nonparticipants), and religious activities (more time spent by participants than nonparticipants). This comparison shows how Beacon Center participants may differ in how they spend their out-of-school time from their other classmates, but does not indicate whether there were differences between participants’ and nonparticipants’ leisure time before the program began.

Evaluation 2: After-School Pursuits: An Examination of Outcomes in the San Francisco Beacon Initiative



Evaluation Description

Evaluation Purpose To answer the following questions about SFBI: (a) Did SFBI create community centers that were welcoming, safe, accessible, and well staffed? (b) Did SFBI offer high quality programming in core areas? (c) How did youth participate in the centers—who came and how often, and what did they do there? (d) Did youth participants in the centers have rich developmental experiences? and (e) Did youth participants experience positive changes in their social well-being and success in school? To examine why and how things happened, the evaluation also focused on the following related questions: (a) What organizational and staff practices contributed to high quality activities? (b) To what extent can youth’s participation in centers be linked to the quality of the activities? (c) To what extent can youth’s developmental experiences be linked to their participation in the Beacon Centers? and (d) To what extent can long-term outcomes be linked to the developmental experiences that youth had in the centers?
Evaluation Design Quasi-Experimental and Non-Experimental: Data were gathered from the first five Beacon Centers that opened between fall 1996 and fall 1998. Three of the five centers operate in middle schools, one is in a high school, and one is in an elementary school. A total of 666 youth completed feedback forms administered to elementary-, middle-, and high school-aged participants at all five Beacon centers during a total of 60 activities that took place in the summer 2000, fall 2000, or spring 2001 sessions. SFBI participants in the three middle school sites were also compared to nonparticipants in the same schools. Eight hundred and thirty-eight middle school students (392 participants and 446 nonparticipants) completed both an initial and an 18-month follow-up survey (76% of the 1,110 youth who completed the initial survey). The middle school youth survey was conducted during the school day, therefore the survey sample comprises a broader range of Beacon participation levels than do the samples of youth who filled out feedback forms or participated in interviews. The feedback form and interview respondents tended to be those who had participated over a longer period of time at the Beacons and were engaged in a greater variety of activities.
Data Collection Methods Document Review: Progress reports, budgets, training materials, and other program documents were collected and analyzed.

Interviews/Focus Groups: Interviews were conducted during twice-yearly visits that were made to each center. Interviews were conducted with Beacon staff, activity providers, community members, school staff, and other stakeholders, including public and private funders, steering committee members, intermediary staff, school district personnel and administrators from city agencies. Approximately 80 staff and community members were interviewed during each site visit, and evaluators also interviewed approximately 35 other stakeholders in institutions that provided crucial support for the initiative. In addition, in-depth interviews and focus groups were conducted with Beacon participants to understand the role the Beacons play in youth’s lives and what participation meant to them.

Observation: Trained observers assessed the quality of the Beacon activities through twice-yearly observations of the developmental opportunities provided to youth at the centers. Information was collected on a total of 112 activities. Evaluators worked with Beacon center staff to select activities across the five core-competency areas of varying quality and with reasonable enrollments. This evaluation measured two out of three of the developmental opportunities listed in the theory of change: supportive relationships and involvement and membership. Opportunities for supportive relationships were measured through the extent to which activities at the Beacon Centers fostered warm adult-youth and peer-to-peer interactions. “Involvement and membership” referred to characteristics such as peer cooperation, decision making, and leadership. The evaluation also measured the structure and management of activities, incorporating other aspects of activity quality, such as how well the adults managed youth’s behavior and the degree to which adults included all youth in the activities. Activities were scored high, medium, or low on each of the dimensions of quality. In addition, evaluators established criteria for a composite rating on the “overall strength” of activities, combined across all dimensions of quality observed. Key thresholds included the number of high and low scores assigned to each activity: A very strong activity had at least four out of six high scores; a strong activity had at least three high and two or fewer low scores; an adequate activity had less than three high scores and two or fewer low scores; and a weak activity had three or more low scores.

Secondary Sources/Data Review: Enrollment and attendance data were collected from a web-based management information system (MIS) that permits all centers to enter enrollment and attendance data for the youth and adults who attend center activities. To assess participation over time, the evaluators examined three “sessions.” One session ran during the summer, starting in mid-June and running through the first 2 weeks of August. Another began around September 1 and ended on December 31, while the third began on January 1 and ended in mid-June. These sessions roughly corresponded to activity schedules at the Beacon centers. In order to examine both variety of activity participation and the potential role of academic enrichment, the evaluators sorted youth’s participation into three categories: (a) youth who participated in educational support activities only, (b) those who did not participate in any educational support activities, and (c) those who participated in educational support activities plus activities in other core areas.

School record data were also collected for the academic years 1998–1999, 1999–2000, and 2000–2001 for participants and nonparticipants in the three middle school sites on gender, ethnicity, free or reduced-price lunch status, grade point averages, suspensions, and school attendance.

Surveys/Questionnaires:
Surveys were conducted with all youth in the three middle schools. These surveys were administered to participants and nonparticipants to measure how they spent their out-of-school time and to document developmental outcomes related to well-being. In addition, Beacon participants completed feedback forms from a subset of activities observed across the five centers to assess youth experiences and relate them to the observed qualities of the activities. Beacon staff surveys were conducted in 2001 to collect information about staff experience, education, and training.

Tests/Assessments: Standardized test data (reading and math SAT 9 Normal Curve Equivalent scores) were collected from school records.
Data Collection Timeframe Data were collected between 1999 and 2001.


Findings:
Formative/Process Findings

Activity Implementation The number of youth activities and services provided by the centers in the five distinct content areas during spring 2001 ranged from 20 to 36.

Centers were especially strong in providing activities in the arts and recreation and educational support areas, but had fewer activities in leadership, career development, and health.

Within the arts and recreation and educational core areas, the sites showed considerable breadth and diversity in programming. For example, arts activities at Chinatown included Chinese lion dancing, sign language, Chinese water painting, Chinese fan dancing, creative expressions, and arts and crafts.

In general, educational activities at the centers could be divided into two categories: (a) academic support, such as tutoring or homework help; and (b) educational enrichment, which incorporated reading, math, science or social studies into projects covering gardening, poetry, cooking, nutrition, and so forth.

The centers’ access to space—both dedicated to the program and shared with the school day staff—influenced the degree to which they could provide informal places for youth and adults to interact. For example, the Chinatown and Sunset Neighborhood Beacon Centers, both of which faced severe space restrictions in their host schools, could not provide much unstructured programming. Despite these challenges, all centers offered a mix of both structured activities and opportunities for informal interaction.

Large majorities of youth across all sites agreed “a little” or “a lot” that they found Beacon activities were challenging and interesting.

The overall quality of half of the activities was strong or very strong, and another 36% were adequate. Only 14% of the activities appeared to be weak.

In more than 50% of the activities, observers rated activity quality as high on such dimensions as how well adults fostered positive interactions and how well they structured and managed the activities. Few activities received low scores on adult-youth interactions, peer interactions, or management and structure.

In only 21% of the activities were 40% or more of the youth observed to have opportunities to take leadership roles and make decisions. Twenty-nine percent of the activities offered youth significant opportunities for decision making, while 44% of the activities had very limited opportunities for youth to make decisions about what would happen in the activity. Poorly rated activities were those in which youth could choose from among a range of options presented by staff but did not have the opportunity to make actual decisions about the activity. For example, youth might have been given opportunities to play one of several board games, but not the opportunity to organize a tournament and establish rules for it. In general, the qualitative youth interviews suggested that when youth were provided with leadership experiences, these experiences were highly valued.

Forty-four percent of all activities scored highly on the extent to which adults fostered peer cooperation, while 22% had low scores.

Although it did not appear that there was an optimum size for activities based on observations, the fewer youth present in an activity, the warmer and more responsive adults generally were.

As the number of staff per youth rose in an activity, there tended to be fewer opportunities for youth to cooperate with one another and take on leadership roles, but more opportunities for positive adult-youth and youth-youth interactions. When adults worked closely with small groups of youth, the focus was often on the relationship between the adult and the youth, not on enhancing cooperative learning environments or on providing leadership opportunities.

Scores for activity structure and management and for warmth of interactions between adults and youth and among peers were fairly similar across activities. Over two thirds of leadership activities had high scores in these three areas, with half or more of other activities receiving high scores. One exception was the relatively small proportion of career development activities that had high scores on peer warmth.

Far fewer activities had high scores on decision making or leadership than on adult warmth and structure and management. Just under half of leadership development activities had extensive decision making opportunities and only a little over a quarter had extensive leadership opportunities.

Very large differences existed across different types of activities with respect to how well adults fostered peer cooperation: 80% of leadership activities exhibited high levels of peer cooperation, whereas only 29% of educational activities did.

Among educational activities, enrichment activities tended to be of either higher or lower quality than the tutoring/homework help programs, while tutoring/homework help activities were more likely to be of medium quality.

Half or more of the educational activities received low scores for leadership and decision making opportunities, and almost a third had a medium level of opportunities for youth to work together. These activities often included one-on-one interaction between adults and youth, but few opportunities for youth to work together. In addition, the extent of the decision making in some activities was to decide what academic subject matter to receive help with on a given day.

Across sites, over 50% of activities had high scores for structure and management, and between 43% and 60% had high scores for adult support. On other dimensions of quality, the variation among the sites was more substantial. For example, about 30% more activities scored high on peer warmth and cooperation in the Community Bridges compared to the Sunset Neighborhood Beacon Center. None of the centers had very many activities with high scores on decision making and leadership.

Youth feedback form data indicated that a large majority of youth across all sites (ranging from 89% to 92%) agreed that they experienced “a little” or “a lot” of support from their peers in activities (e.g., reporting that peers felt comfortable talking to and helping each other). And from 15% to 30% agreed “a lot” with positive statements about the peers in their activities (e.g., reporting that peers in the activity were supportive).

Some middle school youth felt that the older youth with whom they interacted at the Beacons offered an important source of support, either through formal academic mentoring programs offered by the Beacon or through helping to prepare them to deal with high school. In contrast, some middle school youth expressed concerns about high school students at their sites. In one focus group, for example, middle school youth complained that they were sometimes teased or criticized by high school students.

Based on feedback forms, two thirds or more of youth at each center (from 68% to 88%) reported being provided with at least two out of a possible ten types of leadership experiences (such as holding an office in a group, making presentations in front of others or leading other youth in activities); of those, from 30% to 50% reported between six and ten experiences. From 70% to 84% also agreed that the activities provided them with the chance to provide input and make decisions on three to five of the five potential decision making experiences asked about on the form. Thirty percent to 52% reported having four or five of the five potential experiences.

Youth rated their leadership and decision making experiences at the Beacons more favorably than similar experiences at school because of the respectful, collegial way in which youth were treated and because of the autonomy youth were given at the Beacons, which they did not always feel at school.

Seventy percent of youth participants surveyed agreed or strongly agreed that the activities offered by the Beacons were new and different from the typical activities to which they were exposed, either in their communities or schools.

Youth reported more positive developmental experiences (e.g., adult/peer support) in the leadership and arts and recreation activities than in the educational ones.

Observations of warm adult-youth interactions were positively related to youth’s reports of experiencing a supportive adult in the activity.

Observed peer cooperation was positively related to youth’s reports of experiencing supportive peers in the activity.

According to observations, the proportion of youth given opportunities for leadership was positively related to youth’s reported experience of leadership roles in the activity.

Observations of warm adult-youth interactions were positively related to youths’ reports of experiencing challenging experiences in the activity.

The more observed opportunities for peer cooperation that were provided, the better were youth’s reported experiences with adults in the activity.

Although activities that were observed as more highly structured and better managed had higher attendance rates, they also tended to have fewer youth who reported experiencing supportive adults and peers, leadership, decision making, and interest in the activity.
Costs/Revenues The steering committee was heavily involved in developing strategies for ensuring committed long-term funding of the Beacon Centers. When state after school funds became available, it directed the sites to apply for it. In response to concerns among the Beacon directors that the funding would require a level of staffing that they did not have, the steering committee permitted several sites to delay their application to the state. Staff from the DCYF, which chaired the committee, also worked with state legislators to alter the funding.

The total cost per center (operating about 5 days per week year-round) averaged $1,009,990 for 1 year. The costs for core support provided by the initiative funders (the city of San Francisco and private foundations) averaged $311,838 per year. Centers also leveraged substantial amounts of funds and in-kind support from other sources; at minimum, they matched their core funds, and Sunset Neighborhood Beacon Center raised almost three times its core support for the 2000–2001 year.

The cost per day for an individual slot averaged $27, although the variation was large.
Program Context/Infrastructure In interviews and focus groups, youth spoke of the Beacon Centers as physically and emotionally safe spaces. The majority (approximately 85%) of youth completing the feedback forms reported feeling as safe as or safer at the Beacons than at other places where they spent time. Youth noted that the more adults present, the safer they tended to feel. Youth also noted that the presence of friends at the centers contributed to their feelings of safety.

A large proportion of surveyed youth at each center (from 76% to 89%) reported either “a little” or “a lot” that they felt a sense of belonging (i.e., that they were listened to and their ideas mattered). In addition, between 12% and 43% of the youth responded “a lot” to a series of questions measuring their sense of belonging.

Interviews and focus groups revealed youth’s beliefs that their neighborhoods and the neighborhoods surrounding the Beacon Centers were particularly lacking in interesting things for them to do.
Program-School Linkages Initially, Beacon Center relationships with the host schools ranged from excellent to rocky. One Beacon Center, Visitacion Valley, maintained strong, positive relationships with school staff throughout the initiative, with Beacon staff focused on helping the school address students’ educational or behavioral deficits. At the other end of the spectrum, the Community Bridges Beacon and Sunset Neighborhood Beacon Centers had strained relationships with their host schools. In Community Bridges, tensions grew dramatically mid-initiative as the school principal put pressure on the center to provide more academics-pressure that was strongly resisted by the Beacon director.

Beacon staff at two centers invited school staff to Center retreats to discuss future directions of the Beacons. School staff invited Beacon staff to sit in on school meetings, and both Beacon and school staff provided services to each other: Beacon staff worked as teachers’ aides and school staff worked in the Beacon Centers after school.

The relationships between schools and centers improved following the development of a Memorandum of Understanding between the school district and the Beacon centers. This spelled out expectations between the schools and centers and provided staff in some centers with feelings of stability.
Recruitment/Participation Center activities, events, and programs were free in order to ensure that low-income youth and adults could participate, and Beacon Centers offered programs in the early morning, during the day, and in the evening to accommodate the schedules of youth and their families. Centers also increased accessibility by being open almost every day of the week and 52 weeks of the year.

Centers undertook a number of strategies to ensure that their programs were open and welcoming to all students, including signs and banners in highly visible locations both in and outside the schools, flyers listing the Beacon Center offerings by season distributed in the schools and neighborhoods, encouraging school personnel to refer youth to the Centers, and Beacon staff attending meetings of school staff. At one site, staff members distributed information about their programs in the cafeteria during lunch. At another, staff made announcements over the school’s public address system. At a third site, staff maintained a database with all potential students’ names and addresses and mailed out flyers as new programs were developed.

Findings from the spring 2001 survey of middle school seventh and eighth graders showed that almost all the youth (94%) in each of the three schools said they had heard of the Beacon Center in their school, and almost as many knew where it was.

To ensure that the centers would be welcoming to the diverse ethnic populations of the schools and surrounding neighborhoods, schedules of activities and flyers were printed in several languages, among them Russian, Spanish, Cantonese, and Tagalog. The centers also advertised in several languages on the radio and in community newspapers. The centers made themselves welcoming to parents and community adults by hiring staff members who spoke languages spoken in the community. Between 29% and 69% of the staff reported that they used a language other than English with the youth at least part of the time.

In response to questions posed on feedback forms administered during activities at the Beacon, 61% of the youth reported that they missed a scheduled activity at least once (ranging from 41% to 77% at each center). A minority (20%) of youth who had missed activities identified problems getting home after the activity as a barrier. Youth at the Visitacion Valley Beacon were more likely than others to identify this barrier, and it is likely that a lack of access to public transportation was a factor.

In the last year of the evaluation, the number of youth served by each center surpassed the original goal. By the 4th year of operation, during one semester alone (spring 2001) the centers served almost their yearly goal, reaching between 356 and 722 at each center. In addition, center attendance averaged more than 100 youth per day.

In fall 1999, participants in the Beacon Centers located in the middle schools were more evenly distributed by age than were those attending the Beacon Centers in the elementary or high schools. Participation data collected 18 months later indicated that the number of participants who were younger or older than the host school age range decreased at three of the five centers, and the number of youth within the host school age range (and from the host school) increased substantially. As a result, 63% or more of the youth who attended the Beacon Centers in spring 2001 were in the age range served by the host schools, with the exception of Sunset Neighborhood Beacon Center, where 45% of the youth were of middle school age.

Although high school students might have been more likely to attend Beacon Centers located in middle schools than in elementary schools, some older adolescents still expressed reluctance regardless of the age range of the host school. “High school students don’t want to be here with kids, like, way younger than us,” noted one. Older students also perceived that the centers focused their programming primarily on middle school students.

In three Beacon centers, 60% or more of the youth who attended were from the host school. These figures represent increases from fall 1999, when closer to one quarter were from the host schools. In the other two centers, majorities of the youth were from the community rather than from the host school.

In general, the ethnic breakdown of youth served by the Beacon Centers mirrored that of the host schools, although the middle school centers attracted proportionately more Latino and African-American students and fewer Asian students than is reflected in those three schools’ populations. One exception was the Chinatown Beacon Center, where almost all the youth who attended were Asian, even though a substantial number of Latino youth (38%) were bused in from other neighborhoods to attend the host elementary school.

The gender breakdown indicates that approximately equal numbers of males and females were attending three of the five Beacon centers, while at the other two sites, the participants were approximately 60% male.

A comparison of demographic characteristics of middle school participants and nonparticipants indicates that youth were similar on gender and grade level. Differences in how youth fared in school show that Beacon youth had lower grade point averages and math and reading test scores compared to their peers who did not attend the Beacon Centers. Beacon youth also tended to be of lower socioeconomic status, as reflected by the greater proportion of youth who received free or reduced-price lunch at school.

At the initial survey, Beacon and non-Beacon middle school youth were very similar in measures of well-being (i.e., reactions to problems and self-efficacy) and school effort. In measures of developmental experiences, however, those who attended Beacons reported lower levels of peer support and a smaller proportion of their time spent in productive activities. On the other hand, they reported higher levels of nonfamily and family adult support. Both groups reported similar numbers of leadership experiences.

Between 18% and 32% of all Beacon youth at each center participated for three or more sessions.

Youth attended Beacon activities an average of about six times a month, or between one and two visits per week. Community Bridges Beacon youth attended most frequently—in part because a Beacon program was instituted that was considered an extra class period during the school day. Although technically voluntary, Beacon staff at this site reported that the youth who attended thought attendance was mandatory—and attendance rates for enrolled youth were over 90%. Chinatown Beacon Center youth attended almost as often.

Between September 1999 and June 2001, more than half (53%) of all Beacon youth attended educational support activities of some type (e.g., tutoring, educational enrichment). Of those, about half participated only in educational activities, whereas the other half participated in educational activities plus other types of activities. Just under half of the youth (46%) participated in no educational support activities.

Youth appreciated the accessibility of attending a Beacon located on or near the campus of their own school, especially if that site was also in the neighborhood in which they lived. Many liked the fact that they could access a Beacon and its activities without taking a bus or arranging for other transportation. However, a few wished that activities could be held somewhere other than the site at which they already spent most of their day. Data linking responses on the fall 1999 middle school youth survey to future participation (measured by MIS attendance data) indicate that youth who experienced problems getting home from the Beacon were less likely to continue participation in Beacon activities over time compared to youth who did not experience problems.

Youth who, on the fall 1999 survey, reported feeling the most safe at the Beacon after school were more likely than those who felt less safe to continue to attend the Beacon over time. When evaluators looked at both safety and adult support together, however, feeling safe was less important than whether youth found supportive, responsive adults at the center.

The number of supportive adults that participants found early on at the Beacon Center was the most important factor in predicting retention over time, even after taking into account how safe youth felt, whether they had problems getting home from the program, and how new and interesting they found Beacon activities to be (along with controlling for age, gender, and site). The number of supportive adults they found at the Beacon center (as of fall 1999) was also the only factor that differentiated youth who participated for three sessions or more in educational plus other activities from youth with other participation patterns. Evaluators found no relationship, however, between the observed quality of the adult-youth interactions within an activity and attendance rates within that specific activity.

According to youth interviews, the choice in Beacon activities offered was key to attracting youth to participate. Survey data confirmed a strong relationship between youth who rated the activities at the Beacon as providing variety and interest and those who were most likely to continue their participation over a number of sessions. This relationship held after taking into account the youth’s sense of safety and adult supportiveness.

Observation and program attendance data revealed a strong relationship between an activity’s structure and management, and attendance in that activity. The more highly structured and well managed an activity, the more often enrolled participants actually went to the activity. Important factors seemed to be whether staff presented material clearly, organized daily sessions well, and skillfully managed the youth’s behavior.

Observation and program attendance data revealed no relationships between opportunities for peer cooperation, adult-youth and peer interactions, or decision making or leadership provided by adult staff and attendance in activities.

On average, attendance in educational activities was about 11% lower than attendance in arts and recreation activities and about 24% lower than attendance in leadership activities. Many youth noted that they liked having a place to just hang out with friends, and some bemoaned the inflexibility of certain activities, such as tutoring. They reported wanting to talk more with friends or to do other activities once finished with their homework.

Youth interviews and focus groups revealed that friendships were a primary motivation for many youth to spend time at the Beacon, and the lack of same age peers seemed to serve as a deterrent to participation. Data from feedback forms also revealed that, on average, a little over a third of youth who missed an activity indicated they had done so because their friends were not going.
Satisfaction Youth indicated that the things they learned in the centers were relevant to their lives, and they contrasted this with what they were learning at school, which they did not tend to feel was as relevant to their lives.
Staffing/Training The amount and type of staff training varied from center to center, ranging from an average of 1.3 to 3.9 trainings ever attended.

Though staff were more likely than the youth population they served to be White, they were found to be ethnically and racially diverse. Across sites, 41% of staff were White, 29% were Asian/Pacific Islander, 11% were African American, and 9% each were Latino and other.

Approximately 40% of staff across sites reported living in the Beacon Center community in which they worked.

For the most part, youth described their relationships with adults at the Beacons as respectful and reciprocal when asked during interviews and focus groups.

Although each site had a core group of staff (ranging from 25% to 45% of staff) who worked full-time, the majority of staff were part-time instructors who helped run activities.

About half the activities had staff-to-youth ratios between 1:4 and 1:9. From a quarter to half the activities averaged a ratio of 1:3 or smaller. No activity averaged a ratio larger than 1:20. Ratios varied by center, by activity within center, and by senior staff’s goals and expectations for appropriate ratios.

In general, all centers experienced high rates of staff turnover. The longest average tenure for any site was 29 months. Average staff tenure at other centers was about a year, with about two thirds of all staff members in these four centers reporting that they had worked or volunteered for the Beacon for 1 year or less. Centers with fewer staff holding college degrees retained more staff for longer than 1 year. Youth described feeling “abandoned,” “sad,” and as if they were “losing something” when staff left.

According to activity observations, the higher the proportion of full-time staff and of staff with previous experience in a nonprofit youth development agency, the more staff fostered peer cooperation, positive adult interactions, and good structure and management in the Beacon Centers.

The majority of youth (ranging from 84% to 91% at each center) agreed either “a little” or “a lot” on feedback forms that the staff member(s) leading the activity cared about them and spent time talking with them.

Interviews with youth revealed that Beacon staff provided emotional support. They worked to create a space for youth to express their feelings, which youth seemed to value. A little over half (54%) of the Beacon participants surveyed indicated that there was at least one adult at the Beacon Center on whom they could rely for support.

When interviewed, youth described their relationships with adults at the Beacon as more equal than those with adults in other places, such as school or work.

According to activity observations, the more staff per youth in an activity, the better youth’s experiences of adult and peer support in the Beacon activities. Interviews with youth supported this finding; they commented on the importance of the number of staff compared to the number of youth.
Systemic Infrastructure One of CNYD’s most crucial roles was to facilitate communication. It provided staff support for the steering and sustainability committees, called Beacon directors together for monthly meetings, and identified early challenges for stakeholders to address.

Early in the initiative, CNYD provided organizational assistance as some centers struggled to manage the grant and establish programming.

Intermediary staff members ensured that the initiative was represented at important city events, hearings, and conferences. In 2001, CNYD was active in the campaign to reenact the Children’s Amendment—which allotted 3 cents per $100 of assessed property from city taxes to pay for youth and children’s programs. Intermediary staff also spearheaded the creation of public-support materials summarizing the initiative’s work, which were distributed to funders, agencies, and politicians throughout the city.

Intermediary staff supported the Beacon Centers’ efforts to advertise their presence. Those efforts ranged from a citywide Beacon celebration to celebrations sponsored by individual Beacon Centers when the DCYF organized an event to introduce the San Francisco Unified School District’s new superintendent to the initiative in 2001. CNYD ensured that members of the San Francisco City and County Board of Supervisors were invited to events and kept informed of the initiative’s progress, and participated in efforts to inform state-level legislators of the Beacon centers. They also provided staff support for events.

The steering committee worked on systems accommodations and garnering public support throughout the initiative. For example, the Memorandum of Understanding with the school district was seen as a systems accommodation that spelled out specific rights and responsibilities of the Beacon Centers and the schools. Steering committee members worked with the intermediary and the sites to craft an understanding that was acceptable to the schools, the centers, and the school district. The steering committee also worked to secure long-term funding and redirect resources to support the Beacon centers.


Summative/Outcome Findings

Academic Youth who went to Beacon Centers for three sessions and participated in education plus other activities were more likely than nonparticipants to experience increases in school effort, but not other indicators of academic functioning. In addition, youth who participated for three or more sessions in only educational activities reported significantly increased school effort between fall 1999 and spring 2001 compared to nonparticipants.
Youth Development For the middle school students for whom survey data were available, participating in a greater number of sessions in and of itself was not related to positive change in developmental experiences. However, youth who participated in Beacons for longer periods of time (at least two sessions) and in a wider range of activities (educational and other types as well) experienced more gains in nonfamily adult supports and leadership than either nonparticipants or other less engaged participants. Neither peer support nor time spent in challenging activities seemed to increase in relation to Beacon participation.

Youth who went to Beacon Centers for three sessions and participated in education plus other activities were more likely than nonparticipants to show gains in self-efficacy. They were no more likely to have more positive response to social challenges.

Youth in several of the Beacons’ leadership programs reported that they learned to interact with peers in a manner that was respectful and cooperative.

© 2016 Presidents and Fellows of Harvard College
Published by Harvard Family Research Project