The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.
A Profile of the Evaluation of Bayview Safe Haven Program
Profile Information
- Full Text (HTML)
- Request reprint permission
- How to cite
- About the OST Database
Program Description
| Overview | The Bayview Safe Haven program (BVSH) is an after school program for at-risk youth ages 10–17. It is designed to help youth stay in school and out of the criminal justice system, while positioning them for responsible adulthood and improving the quality of life in their families and community. In a community with a dearth of programs for at-risk youth, BVSH is designed to provide a hub of structured activity and to serve as a central site where public and private collaborators can channel resources and services to the youth and families of San Francisco's Bayview/Hunter's Point neighborhood. |
| Start Date | November 1997 |
| Scope | local |
| Type | after school |
| Location | urban |
| Setting | recreation center |
| Participants | elementary through high school students |
| Number of Sites/Grantees | one |
| Number Served | 126 (over first two years, fall 1997–spring 1999) |
| Components | BVSH programs are grouped into four categories in which youth participate during after school hours: academic, vocational, recreational, and life skills and community service.
After San Francisco League of Urban Gardeners took over the management of the program, BVSH expanded its hours to 1–9pm Monday through Friday, with participants beginning to be transported home at 8pm. Prior to the transition to SLUG, the BVSH operated from 3–9pm. |
| Funding Level | For fiscal year 2002–2003 the total operating budget for the BVSH was $459,415. |
| Funding Sources | California State Board of Corrections Juvenile Crime Enforcement and Accountability Challenge Grant I; Evelyn & Walter Haas Foundation |
| Other | BVSH is one of six components of San Francisco's Juvenile Justice Plan, also known as the Local Action Plan (LAP), which together form a cross-sector, cross-systems network of interventions which DSF refers to as the CIRCLE of Care, or Coalition to Implement Revitalized Communities, Lives, and Environments. The Delancey Street Foundation (DSF) supervised BVSH's three-year implementation. In 2000, SLUG became the new fiscal agent for BVSH, charged with maintaining the principles and design developed by DSF. |
Evaluation
| Overview | The Mayor's Criminal Justice Council (MCJC) retained BTW Consultants, Inc., an independent research and consulting firm, to conduct a three-year comprehensive evaluation of the BVSH program. The evaluation was conducted in collaboration with LaFrance Associates, LLC. |
| Evaluators | Steven LaFrance and Nancy Latham, LaFrance Associates Fay Twersky, Eileen Foley, Linda Lee, and Cynthia Bott, BTW Consultants, Inc. |
| Evaluations Profiled | A Safe Place for Healthy Youth Development: A Comprehensive Evaluation of the Bayview Safe Haven |
| Evaluations Planned | The Community Development Institute is currently conducting ongoing program evaluation with the Bayview Safe Haven. |
| Report Availability | LaFrance, S., Twersky, F., Latham, N., Foley, E., Bott, C., & Lee, L. (2001). A safe place for healthy youth development: A comprehensive evaluation of the Bayview Safe Haven. San Francisco, CA: BTW Consultants & LaFrance Associates. |
Contacts
| Evaluation | Steven LaFrance, MPH LaFrance Associates, LLC 1242 Market Street, 3rd Floor San Francisco, CA 94102 Tel: 415-241-0605 Fax: 415-252-1068 Email: steven@lfagroup.com |
|
| Program | Lena Miller Bayview Safe Haven 1395 Mendell St. San Francisco, CA 94124 Tel: 415-822-8894 Fax: 415-822-7016 Email: bayviewsafehaven@aol.com |
|
| Profile Updated | April 29, 2003 | |
Evaluation: A Safe Place for Healthy Youth Development: A Comprehensive Evaluation of the Bayview Safe Haven
Evaluation Description
| Evaluation Purpose | To test hypotheses pertaining to two units of analysis: (1) individual youth participants and (2) the surrounding neighborhood. For individual youth participants, the evaluation tested hypotheses predicting that participants would exhibit lower levels of school suspensions, juvenile crime recidivism, seriousness of delinquent behavior, and wardship of the court status, fewer school expulsions and arrests, and higher levels of probation completion than a nonparticipating comparison group. For the surrounding neighborhood, the evaluation tested hypotheses predicting that BVSH's census tract would show a greater reduction in juvenile crime than comparison census tracts. |
| Evaluation Design | Quasi-Experimental and Non-Experimental: For the individual youth participant component of the evaluation, a group of 126 youth ages 10–17 were designated as the treatment group. These were youth who not only came to BVSH, but also “joined” the program. A youth joins BVSH by coming to the program, meeting with a staff person to review and to agree to abide by the rules, signing up for one of the program's activities (such as the bike repair program), showing up for that program or activity, and having an assessment interview with staff. Evaluators also constructed a comparison group using a stratification process, matching comparison group youth to treatment group youth based primarily on the referral source of the treatment group youth. If the police or probation systems referred a treatment group youth, a comparison group youth was selected from the 1997 Probation Department database. If the San Francisco Housing Authority referred a treatment group youth, a comparison group youth was selected from that same housing authority. If a community-based organization or a participant's friend or family referred a treatment group youth (or if the participant was self-referred), a comparison group youth was selected from the San Francisco Unified School District's (SFUSD) Master Student Roster. Comparison group youth were further matched to their treatment group counterparts by age, gender, juvenile justice history (ever involved with the Probation Department), and whether the youth had a school performance problem (standardized test score below 45% on reading or math). There were no significant differences between treatment and comparison groups in terms of age, sex, race/ethnicity, ward of the court status, petitions sustained for felony offenses, informal probation status, and court work program assignment status. Moreover, the vast majority of both treatment and comparison group youth were not performing at grade level according to standardized test scores (97.6% of treatment youth vs. 93.4% of comparison youth, p=.182). Treatment group youth were, however, significantly more likely than comparison group youth to have a documented history of abuse or neglect (53.2% vs. 33.6%, p=.001). A higher percentage of treatment group youth (10.3%) than comparison group youth (5.6%) had been referred to SFUSD's Drop Out Prevention Program prior to intake, although this difference was not statistically significant (p=.125). A significantly higher proportion of treatment youth had been referred to the Juvenile Probation Department than comparison youth (73.4% vs. 59.2%, p=.013). There are three time periods for data analysis for the individual youth participant component, the “before intake period,” the “intervention period,” and the “follow-up period.” Within the follow-up period, there are three subdivisions of six months each; thus, the follow-up period spans 18 months total after exiting the program. For the neighborhood impact component, a pretest/posttest design was used with one treatment and four comparison neighborhoods. The Bayview/Hunter's Point census tract (in which BVSH is located) was considered the treatment neighborhood, while four other census tracts served as a comparison group, selected based on either proximity to the treatment tract or comparable population and crime characteristics as the treatment tract. The evaluator urges caution in interpreting the neighborhood impact component, given that the census tract is an imperfect unit of analysis for community level impacts (since participating youth live both within and outside the treatment tract, and since the intervention functions at the individual, not the community, level). Furthermore, different histories of the census tracts could be a threat to validity, and too few years of data were available to conduct powerful statistical tests. |
| Data Collection Methods | Document Review: BVSH provided the following: daily sign-in sheets tracking program attendance, information on treatment interventions youth received while in the program, and a log tracking participating youth's case management referrals and status at the time of exiting the program (Successful, No Fault—left program in good standing, but prior to close of six months of intervention, and Not Successful). Interviews/Focus Groups: BVSH conducted on-site face-to-face intake interviews with all new youth, typically after the youth had attended for two or three days. These interviews asked about type and safety of current housing and living situation, family and household composition, current school enrollment and attendance, gang membership and affiliation, perceptions of available social support, drug and alcohol use, familial involvement in the justice system, familial substance use, self-identified areas of strength, and areas where youth would like to learn and grow. Evaluators also conducted interviews with 38 BVSH youth and 16 key informants (police, program staff, and community-based organization staff), asking respondents about their perceptions of the program and its impacts on their lives and on the surrounding community. Secondary Source/Data Review: SFUSD provided data on school suspensions, expulsions, and school attendance. SFPD provided access to seven years (1993–1999) of San Francisco juvenile crime incident data (all incidents where the suspect's age was between 10 and 17 were included in the extracted dataset). San Francisco Juvenile Probation Department provided individual youth's arrest and probation histories from the Juvenile Justice Information System database; for youth who turned 18 during the course of the study, criminal activity data were obtained from the adult justice system in San Francisco. A total of 5,197 cases of juvenile crime incidents for juveniles ages 10 to 17 were extracted for the five census tracts in question. The Department of Human Services/Child Protective Services provided data on any Suspected Child Abuse Reports (SCARs) filed on any treatment and comparison group youth. Surveys/Questionnaires: BVSH surveyed 100 youth participants on selected risk factors prior to the evaluation for program planning purposes, primarily to enhance understanding of the type of youth served by BVSH. |
| Data Collection Timeframe | Data were collected between 1998 and 2000. |
Findings:
Formative/Process Findings
| Program Context/Infrastructure | San Francisco's Bayview/Hunter's Point neighborhood exhibits a range of risk factors and needs among youth, such as school failure, behavioral problems, crime-involved families, substance abuse, mental health issues, and prior criminal convictions. Bayview/Hunter's Point had the second highest volume of juvenile crime in San Francisco. Area crime patterns show that youth ages 10–17 are most likely to be involved in crimes in the daytime immediately after school, between 3 and 4pm when they are out of school, but before parents come home from work. |
| Recruitment/Participation | The BVSH set out to serve 120 youth over a two-year period and the program exceeded this goal, serving a total of 126 youth during that time. Youth participants came to the BVSH through self-referral (21.4%), referrals from the Juvenile Probation Department and police (36.5%), the LAP's Community Assessment and Referral Center (16.7%), friends and relatives (word of mouth, 12.7%), and the San Francisco Housing Authority (10.3%). The length of time in days that youth were involved with the BVSH ranges broadly from about two weeks (12 days) to almost two years (583 days). On average, youth were involved for 150 days. BVSH youth range in age from 10–17 years old, with the largest groups being 12–13 year olds (37.3%) and 14–15 year olds (28.6%). A little over 70% of participants are male. Almost all participants are African American (96.8%). Three-quarters of BVSH youth live with one birth parent or both birth parents, while 22% live with a relative other than their parents. Over one-quarter of BVSH youth live with four or more siblings (21% have three siblings, 20% have two siblings, 20% have one sibling, and 12% have no siblings). Over one-third of BVSH youth (36.5%) reported having a drug use problem at the time of intake. Less than one-quarter (23%) reported having an alcohol use problem at intake. Over one-third (38.9%) used either drugs or alcohol at intake. In comparison, the 100 youth surveyed prior to the evaluation demonstrated markedly higher levels of substances, with 70% reporting using any substances, and 60% reporting using both alcohol and drugs. Approximately 42% of BVSH youth have current or past affiliation with gangs (which includes either having been a member or “hanging out” with gang members) prior to intake. Male youth (49.4%) were more than twice as likely to have current or past gang affiliation than girls (22.9%) (p=.006). Also, 16–17-year-old youth (61.9%) were more likely to have current or past gang affiliation compared to 14–15 year olds (48.5%), 12–13 year olds (42.2%), and 10–11 year olds (5.3%) (p=.001). Approximately one-third of BVSH youth reported parental alcohol use and one-third reported parental drug use. Three in ten BVSH youth report sibling alcohol use and one in five report sibling drug use. Approximately one-third of BVSH youth reported that a parent had been arrested or had been in jail. More than one-fifth of BVSH youth reported that a sibling had been involved in the criminal justice system. Over one-third of BVSH youth reported not having a trusted adult in their lives with whom they could talk prior to intake. In comparison, the 100 youth surveyed prior to the evaluation showed a much higher (75%) percentage reporting that they did not have an adult in whom they could confide. Prior to intake, 51.8% of BVSH youth reported not having a place to go when “things were not going well.” Just over one in five BVSH youth had changed their residence at least once in the year prior to their intake interview. Just over one-quarter of BVSH youth had been institutionalized at least once in the year prior to their intake interview. Nearly 15% of BVSH youth were not enrolled in school at the time of intake, despite their young age. |
Summative/Outcome Findings
| Academic | Although sample sizes are small (n=18, treatment group; n=5, comparison group), 100% of treatment youth with a history of school suspensions were not suspended during the intervention period, while 100% of comparison group youth with a history of suspension were suspended during the intervention period (p<.001). For those youth without a history of suspension, no treatment youth were suspended during the intervention period, while 10% of comparison youth were suspended during the intervention period (p=.001). For all youth suspended at least once at either intake or at the intervention stage, treatment youth decreased the number of times they were suspended by 1.0 times, while comparison youth increased the number of times they were suspended by 0.4 times, a statistically significant difference (p<.001). There were no statistically significant differences in expulsion during the intervention period for those with no history of expulsion, although the incidence of expulsion was extremely low in both groups (1.6% for treatment youth vs. 3.3% for comparison youth). A higher percentage of treatment youth with attendance problems prior to intake had no attendance problems during the intervention period (77% vs. 57%), although this difference was not statistically significant (n=13, treatment; n=7, comparison; p=.179). For those with no attendance problems prior to intake, no one in either group had an attendance problem during the intervention. |
| Community Development | For the neighborhood component of the study, data showed that juvenile crime did decline sharply in the experimental census tract (from a rate of 28% in 1993 to a rate of 10% in 1999). The experimental tract started with the highest level of crime and ended among the lowest level of crime in the five census tracts. Crime dropped substantially in most of the tracts, however, making it difficult to infer that the intervention caused the decline. |
| Prevention | Approximately 44% of comparison group youth recidivated from before intake to the intervention period, while 19.8% of the treatment group recidivated (p=.001). Controlling for gender and age in a regression model did not weaken the effect of treatment. Approximately 59% of comparison group youth had at least one arrest during the follow-up period, while 40% of treatment youth had at least one arrest during this period (p=.01). This effect declined slightly when controlling for gender and age, but was still marginally statistically significant (p<.1). There were no statistically significant differences in the mean number of arrests experienced by treatment and comparison group youth, most of those who were arrested were arrested only once. Successfully completing the BVSH program was not significantly related to the probability of being arrested during the follow-up period. BVSH youth who came to the program voluntarily had a lower rate of arrest at the intervention period than those whose presence was mandated to attend as a condition of probation (3.8% of voluntary attendees vs. 29.8% of mandated attendees, p=.008). Among those with at least one previous arrest, significantly fewer voluntary attendees experienced an arrest during the follow-up period than mandated attendees (8% vs. 61%, p<.001), a relationship which was not weakened by controlling for gender and age. For those who have had a petition sustained for an arrest either before intake and/or during the intervention period, treatment youth demonstrated significantly more positive outcomes than the comparison group in terms of change in seriousness of crime (p<.01). Seriousness is measured by an index ranging from 1 – “violation of probation or court order” to 10 – “felony – violent offense.” Thus, the outcome is change in this seriousness index. For those who have had a petition sustained for an arrest either before intake and/or during the follow-up period, treatment youth demonstrated significantly more positive outcomes than the comparison group in terms of change in seriousness of crime (p<.01). A significantly higher percentage of treatment youth had their wardship of the court status terminated during the follow-up period (27.3% vs. 5.0%, p=.027). Adding controls for gender and age, however, did weaken the relationship to where it was not statistically significant. No comparison group youth completed probation by the time of the follow-up period, while 4 of 30 treatment group youth did (13.3%). Logistic regression controlling for gender and age, however, revealed that this relationship was not statistically significant. Among youth with no arrests prior to intake, a significantly smaller percentage of treatment youth were arrested during the intervention period (12.1% vs. 36%, p=.02), a relationship that was not weakened after controlling for gender and age. A smaller percentage of treatment youth with no arrests prior to intake had been arrested during the follow-up period, although this difference was not statistically significant (28.1% vs. 44%, p=.15). |