Search Publications | The Evaluation Exchange | FINE Network | OST Database

The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.

A Profile of the Evaluation of Say Yes First

Program Description

Overview	Say Yes First—to Rural Youth and Family Alcohol/Drug Prevention (SYF), a rural, multi-component, school-based, drug-prevention project in northern Colorado, provided primary and secondary prevention services for youth as they progressed from fourth through eighth grade. This five-year demonstration project focused on protective and resiliency factors such as academic success and personal involvement in positive drug-free experiences in youth's families, schools, and communities. The project staff designed educational and recreational activities as alternatives to alcohol, tobacco, and other drug (ATOD) use for youth.
Start Date	1991 (completed 1996)
Scope	local

Type	after school, summer/vacation, before school, weekend, comprehensive services
Location	rural
Setting	public school
Participants	elementary through middle school students (fourth through eighth grade)
Number of Sites/Grantees	four school districts
Number Served	859 (1991–1996)
Components	SYF's approach included school team training, substance abuse prevention programs, and case management of high-risk youth and families. To create positive social influences, low-risk students were also included in the intervention. School Team Training Project staff conducted two-day school district team trainings each fall to acquaint school personnel with the program. Each school district team consisted of a school administrator, teachers from the designated grade, parent representatives, other interested school staff, the program manager, and the case manager. Substance Abuse Prevention Program School staff and a program manager in each district delivered a substance abuse prevention and education program that included: comprehensive health education for all students, academic improvement and enhancement programs for at-risk students, parent-education programs and involvement strategies, alternative activities for youth and families, SYF councils, and youth-leadership trainings. Health Education Programs The comprehensive health education program was designed to help youth develop healthy lifestyles and resiliency factors such as self-acceptance and problem-solving skills. It entailed specialized healthy lifestyle curricula, including the nationally validated Growing Healthy¹ and Teenage Health Teaching Modules² curricula for all students in all grades. Academic Programs Students who needed academic assistance were recruited into school programs in which small groups of students participated in structured academic activities such as before, during, and after school academic tutoring and mentoring programs with an emphasis on math, science, and/or language-art skills. Summer academic-enrichment programs in grades four through six were held daily for two to four weeks. Students also participated in a two-day creative, summer academic enhancement program designed to help in writing, science, math, communication, and envisioning a positive future for themselves (including the possibility of attending a local college or university). Parent Education Programs and Involvement Strategies Parent education programs, such as the Nurturing Parenting Program,³ Preparing for the Drug-Free Years,⁴ Cara y Corazon (Face and Heart),⁵ and Padres Con Poder (Parents With Power),⁶ focused on: substance abuse education, parenting skills, family communication, career development, and use of community resources. Each program manager was assisted by a Parent Advisory Committee (PAC), made up of parents and teachers. PAC meetings were held quarterly and usually included an educational program for parents, as well as recreational or educational activities for their children so entire families could attend. Alternative Youth and Family Activities Alternative, drug-free school and recreational activities for high-risk youth and their families included weekly, noncompetitive summer softball practices (grades four through six), monthly noncompetitive family sports tournaments, and BBQs. Local school district sports camps (e.g., Tae Kwon Do, basketball, swimming, cheerleading, roller skating) in grades five and six emphasized having fun, developing interpersonal skills, acquiring sport/activity skills, and building team cooperation. Other activities included evening entertainment programs, a community health walk, artwork, skits, plays, and musicals. Family-oriented summer activities included monthly field trips to the zoo, museums, a national park, evening concerts at the local university and professional baseball games on the weekend or in the evening. Some trips were organized as student rewards for previous program participation or improvement in grades. School bus transportation was provided for families to attend recreational activities. In grades five and six, SYF students organized service projects to contribute back to their local communities. SYF Councils SYF councils were implemented in grades six and seven and consisted of 8 to 25 high- and low-risk class representatives per school district each year. The councils provided a forum for students to help plan and deliver SYF programs, recruit students and parents into SYF activities, and disseminate information about SYF to their classes. Youth-Leadership Trainings Youth Leadership Training Conferences were organized for students at a local university. These one-day conferences included get-acquainted games and inspirational leadership talks by local professionals. High school student leaders also discussed with small groups of students the importance of staying drug free, doing well in school, and being involved in school programs, as well as how to prepare for the transition to high school and for future careers. About 60 students participated in each conference. Case Management A case manager in each district handled a caseload of 12 to 17 high-risk youth and their families who were referred by school staff; they were recruited by personal invitation with case manager home visits. The case manager conducted initial interviews, made home visits to assess youth and family needs, recommended a prescriptive plan, which included referral to school and community services (such as counseling, social services, and substance abuse treatment), and provided follow-up to help families progress. To ensure quality control, documented plans and progress reports were reviewed. Grant administrative staff and an ATOD consultant held monthly case management meetings to educate case managers about community resources for families and for referrals. ¹ National Center for Health Education. (1991). Growing healthy: A comprehensive health school health education curriculum [Grade 4–6 curricula]. New York: Author. ² Education Development Center. (1983–1992). Teenage health teaching modules [Teacher Curriculum Guides]. Newton, MA: Author. ³ Bavolek, S., & Bavolek, J. (1989). The nurturing program. Park City, UT: Family Development Resources. ⁴ Developmental Research and Programs, & Roberts, Fitzmahan & Associates. (1988). Preparing for the drug (free) years: A family activity book (2nd ed.). Seattle, WA: Comprehensive Health Education Foundation. ⁵ Tello, J., Barrera, R. M., Cardenas, F., Somyak, B. L., & Lester, M. E. R. (1994). Cara y corazon: A family-strengthening, rebalancing and community mobilizing process. Austin, TX: Corporate Fund for Children. Canek Media Productions. (1994). Padres con poder: Parent empowerment program [Video program] ⁶ Canek Media Productions. (1994). Padres con poder: Parent empowerment program [Video program]. Houston, TX: Author.
Funding Level	$2.2 million (total funding 1991–1996)
Funding Sources	The U.S. Department of Health and Human Services Center for Substance Abuse Prevention (CSAP)

Evaluation

Overview	A longitudinal evaluation of the duration of the SYF program examined academic and prevention outcomes for a cohort of student participants in the SYF program as they progressed from fourth through eighth grade. A three-year follow-up of program participants was conducted in 1999 when students were in eleventh grade. In addition, evaluators conducted a cost-benefit analysis of the project.
Evaluators	Victor Battistich, External Evaluator and Deputy Director and Associate Professor, Educational Psychology, Research and Evaluation, University of Missouri-St. Louis Rachel Barton, Evaluation Associate, and Richard J. Delaney, Evaluation Assistant, University of Northern Colorado Administrators Kathleen J. Zavela, Director, and Ruben Flores, Assistant Director, University of Northern Colorado Consultant B. J. Dean, Project Consultant and Director, Island Grove Treatment Center, Greeley, Colorado
Evaluations Profiled	Say Yes First: A Longitudinal, School-Based Alcohol and Drug Prevention Project for Rural Youth and Families Say Yes First: Findings from the High School Follow-Up Assessments
Evaluations Planned	none (project completed)
Report Availability	Zavela, K. J., Battistich, V., Dean, B. J., Flores, R., Barton, R., & Delaney, R. J. (1997). Say Yes First: A longitudinal, school-based alcohol and drug prevention project for rural youth and families. Journal of Early Adolescence, 17(1), 67–96. Zavela, K. J., Shaw, A. R., & Dean, B. J. (1998). Cost-benefit analysis of the Say Yes First-to Rural Youth and Family Alcohol/Drug Prevention Program. San Juan, PR: XVI World Conference on Health Promotion and Health Education. Zavela, K. J., & Battistich, V. (2001). Say Yes First: Findings from the high school follow-up assessments. Greeley, CO: Author.

Contacts

Evaluation	Kathleen J. Zavela, M.P.H., Ph.D., C.H.E.S. Professor of Community Health College of Health & Human Sciences University of Northern Colorado Department of Community Health and Nutrition Gunter Hall 2360 Greeley, CO 80639 Tel: 970-351-1516 Fax: 970-351-1489 Email: kathy.zavela@unco.edu
Program	Kathleen J. Zavela, M.P.H., Ph.D., C.H.E.S. Professor of Community Health College of Health & Human Sciences University of Northern Colorado Department of Community Health and Nutrition Gunter Hall 2360 Greeley, CO 80639 Tel: 970-351-1516 Fax: 970-351-1489 Email: kathy.zavela@unco.edu
Profile Updated	August 12, 2003

Evaluation 1: Say Yes First: A Longitudinal, School-Based Alcohol and Drug Prevention Project for Rural Youth and Their Families

Evaluation Description

Evaluation Purpose	To examine whether involvement in SYF was associated with significant reductions in students' drug-use prevalence over time and improvements in academic performance.
Evaluation Design	Quasi-Experimental: The sample consisted of a cohort of students in the graduating class of 2000 in four rural school districts in northern Colorado who participated in SYF from fourth through eighth grade. Students were involved voluntarily and with parental permission. Three of the four districts had community populations less than 2,000 and one had a population of about 5,000 according to 1990 census data. All four communities have an agricultural base, but are “bedroom communities” for nearby cities. Students were predominantly European American, although a high percentage of students were Hispanic (primarily of Mexican descent)—about 18% to 48% in each district. The total sample consisted of 859 participants (the five-year cohort), who were involved in SYF in at least one of the five years, and were followed through the program from fourth to eighth grade. The results of this report focus on a longitudinal subsample of those who participated during the first four years (fourth through seventh grade) since final summative data on academic performance and self-concept skills were not available at the time of this report. A total of 785 students participated in at least one of the first four years (the four-year cohort). An initial cohort of 418 students in fourth grade completed surveys. Of these, 395 (94%) participated for all four years and thus were part of the four-year subsample for the analyses reported (the four-year longitudinal subsample). Six percent of the original cohort of fourth graders were not available for surveying for the entire study period; some of the students switched schools, some joined the schools beyond fourth grade, and a very small percentage dropped out. Preliminary analyses indicated that the four-year longitudinal subsample did not differ systematically from the rest of the five-year cohort or four-year cohort with respect to gender, ethnicity, risk status, or prevalence of ATOD use in the fourth grade. The assessment of program effects was complicated by the absence of a contemporaneous no-treatment comparison group. Consequently, two approaches were taken to examine SYF's effectiveness. For the first set of analyses, differences in program participation levels were related to changes in academic performance and ATOD use over the first four years (fourth through seventh grade) within the longitudinal subsample. Evidence of program effectiveness would be indicated if increases in program participation were associated with improved academic performance and decreased ATOD use. Student risk level was included to determine if the effects of program participation level were moderated by level of risk. Thus, a series of 3 (Low, Moderate, and High Participation) x 3 (Low, Moderate, and High Risk) analyses of covariance (ANCOVA) were conducted. The dependent measures were change scores; hence, initial (fourth grade) scores on measures of academic performance and ATOD use were controlled. Student gender and ethnicity were included as covariates to increase the precision of effect estimates. In a second set of analyses, the entire SYF cohort that was still in the sample in eighth grade (1995–1996, n = 430) were compared to eighth graders in the same four school districts during the three preceding school years: 1992–1993 (n = 401), 1993–1994 (n = 370), and 1994–1995 (n = 516) who received parental consent as no-treatment comparison groups for examining ATOD use. Evidence of program effects would be provided if there were a clear pattern of stable or increasing ATOD use among the preceding cohorts, coupled with a clear deviation from trend (i.e., decrease in use) for the SYF cohort.
Data Collection Methods	Interviews/Focus Groups: School staff were interviewed during school team meetings regarding risk and resiliency changes they observed in the students. Secondary Sources/Data Review: Grades and attendance rates were obtained from school district records. Student participation in SYF was documented by monthly tracking of each SYF activity. Program and case managers recorded the frequency of program participation in monthly contact units for each intervention activity. Students were classified into three participation groups of approximately equal size based on the total number of program activities in which they participated during the project: low (n = 125, up to 20 activities), moderate (n = 138, between 21 and 30 activities), and high (n = 131, over 30 activities). Virtually without exception, students in the high participation group participated more often in each type of activity than did students in the moderate or low groups, and students in the moderate group participated more than did students in the low participation group. Surveys/Questionnaires: During the fall of each year, an internally developed Drug Use Survey was administered to students, which assessed lifetime drug use and drug use during the last 30 days. This survey was administered to SYF eighth graders and to no-treatment eighth graders. To ensure confidentiality, only student identification numbers were used for data collection and analyses. Students were asked about their use of cigarettes, chewing tobacco, alcohol, marijuana, inhalants, cocaine/crack, steroids, over-the-counter drugs, and prescription drugs. Three basic questions were asked about each drug: (1) Have you ever used the drug? (2) At what age did you first use it? and (3) How much have you used it in the past 30 days? For cigarettes, alcohol, and chewing tobacco, students were asked how they obtained the drug. Students who indicated use of alcohol were also asked how much alcohol they usually consumed at one time. At the end of each school year, teachers involved in the program and school principals were asked to complete a survey regarding their involvement in SYF, student changes they attributed to SYF, and project strengths and weaknesses. Tests/Assessments: Student academic-achievement test scores in reading and math were obtained from school district records. Students' ATOD risk level was assessed using a risk-factor-assessment form, which weights drug-risk factors⁷ identified by CSAP and in literature on drug risk factors on a scale from one to five. Managers used this form in a student-interview format. Students were classified into three groups based on their weighted risk-factor scores: low-risk (n = 110, scores of 0); moderate-risk (n = 131, scores of 1 to 7); and high-risk (n = 154, scores of 8 or higher). The predominant risk factors identified for the four-year subsample were: 49% school dropout risk (including academic failure), 29% family-management problems, 23% antisocial behavior, and 12% student, friend, or parental use of drugs. ⁷ Lorion, R. P., Bussell, D., & Goldberg, R. (1991). Identification of youth at high risk for alcohol and other drug problems. In E. N. Goplerud (Ed.), OSAP Prevention Mongraph-8: Preventing adolescent drug use: From theory to practice (DHHS Publication No. (ADM) 91–1725, pp. 53–90). Rockville, MD: US Department of Health and Human Services.
Data Collection Timeframe	Data were collected during all five school years of the project, 1991 through 1996.

Findings:
Summative/Outcome Findings

Academic

Participation level for the four-year longitudinal subsample was significantly associated with trends in reading grades, p = .005. High-participation students showed less decline in reading grades over time (mean = -.08, p < .10, Scheffé post-hoc comparison) than did either moderate (mean = -.37) or low-participation students (mean = -.35).

Participation level for the four-year longitudinal subsample was significantly associated with achievement test score trends in reading, p = .04 and math, p = .025. High-participation students improved their achievement test scores more over time (p < .10, Scheffé post-hoc comparison) in reading (mean = 3.88), and math (mean = 4.10) than did moderate-participation students (mean = 3.28, 3.49, for reading and math, respectively). Low-participation students had intermediate scores on both achievement measures (mean = 3.77, 3.87 for reading and math, respectively).

Overall, for the four-year subsample, the mean reading and math achievement test scores were just below fourth grade level in the fourth grade and improved to above seventh grade level in the seventh grade.

No significant associations were found between participation level and: math grades, science grades, language arts grades, grade point average, or absences.

Prevention

Program participation level was significantly associated with lifetime prevalence of use of “other” drugs (i.e., cocaine/crack, steroids, other illegal drugs, prescription and over-the-counter medications), p = .004.

High-participation SYF students reported less of an increase (p < .10, Scheffé post-hoc comparison) in use of other drugs over time (mean = .04) than did moderate-participation students (mean = .23). Low-participation students increased their use of other drugs by an intermediate amount (mean = .11).

Program participation level was significantly associated with the amount of alcohol typically consumed when drinking, p = .092.

Over time, low-participation students reported a greater increase in their amount of alcohol consumption (mean = .42) than did moderate- (mean = .22) or high-participation students (mean = .27).

Participation level was not found to be significantly associated with lifetime use of alcohol, marijuana, or use of any drugs overall; for last 30 days use of inhalants, marijuana, or other drugs; or for amount of alcohol consumed.

Risk Level x Participation Level interactions were found for changes both in frequency of tobacco use (either cigarettes or chewing tobacco) in the last 30 days, p = .039, and in frequency of use of any drug in the last 30 days, p = .056. In each case, for high-risk students, increased participation was associated with decreased drug use; for moderate-risk students, increased participation was associated with increased drug use; and for low-risk students, there was no systematic relationship between participation level and drug use. No other significant Risk Level x Participation Level interactions were found.

Analyses of eighth grade cohort differences indicated that the program was associated with significantly reduced lifetime use of alcohol (p < .001) and use during the past 30 days of chewing tobacco, alcohol, crack/cocaine, steroids, prescription drugs, and over-the-counter medications (p < .001 for each). In each case, prevalence increased linearly from 1992–1993 to 1994–1995 and then decreased for the SYF cohort in 1995–1996.

SYF students at the eighth grade level did not differ significantly from any of the previous eighth grade cohorts for “ever” use of marijuana, inhalants, cocaine/crack, steroids, prescription drugs, or over-the-counter medications, or for use in the last 30 days of cigarettes, marijuana, or inhalants.

Youth Development

School staff interviews indicated that the most noticeable differences in the students over the first four years of the project from previous years of students who were not in the project were peer bonding and cooperation among the SYF students. The teachers and principals noted that SYF students were more cooperative with each other and worked together more as a group.

Evaluation 2: Say Yes First: Findings From the High School Follow-up Assessments

Evaluation Description

Evaluation Purpose	To determine the long-term impact of SYF on participants.
Evaluation Design	Quasi-Experimental: A follow-up study of students who participated in SYF in grades four through eight and comparison students was conducted when students were in the eleventh grade; 120 former program students and 49 comparison students from four high schools and an additional 87 comparison students from a fifth high school (136 comparison students total) were assessed. The fifth school was comparable to the SYF schools in terms of its rural location and demographic characteristics. Years of participation in SYF among program students ranged from one to five, with almost half (48%) reporting participation for all five years. The overall sample was 63% female, 79% white, and 77% from two-parent families. Students were an average of just under 17 years old (mean = 16.83 years) and almost all (97%) were in the eleventh grade. Program and comparison samples did not differ by gender, age, ethnicity, or family composition, nor in the proportion who indicated that their school courses included learning how to: feel good about themselves, get along with others, or make better decisions. However, a significantly higher proportion of comparison (63%) than program students (39%) indicated that their school courses included learning about ATOD (p < .001). Conversely, program students indicated that they had participated in more prevention programs (other than SYF) in their lifetimes than comparison students (p < .01), and that they had been involved in more extracurricular activities than comparison students during the past year (p < .001). Finally, more comparison students had a part- or full-time job (72%) than program students (52%) at the time of follow-up (p < .002), although among those who were employed, comparison and program students did not differ in how long they had been working. Data were analyzed controlling for gender, ethnicity, and age.
Data Collection Methods	Surveys/Questionnaires: The CSAP National Youth Survey (Follow-up Questionnaire)⁸ was used to obtain information about ATOD use and associated risk and resiliency factors for the participant and comparison groups. ⁸ EMT Associates. (2000). National cross-site evaluation of high risk youth programs, final technical report. Folsom, CA: Author.
Data Collection Timeframe	Data were collected in the spring of 2000.

Findings:
Summative/Outcome Findings

Academic

Program students reported significantly higher grades in school than comparison students (means = 3.18, 2.70, respectively; p<.001, effect size = .54).

SYF students reported missing significantly fewer days of school in the two months preceding follow-up than comparison students (means = 1.26, 1.75 respectively; p < .02, effect size = -.30).

SYF youth scored significantly higher than comparison youth on a measure of positive attitude toward school (means = 3.41, 3.22, respectively; p < .03, effect size = .28).

Family

Program students had significantly fewer disagreements or arguments with their parents than comparison students (means = 3.04, 3.25, respectively; p < .08, effect size = -.22).

Program students scored significantly higher than comparison students on measures of communication and involvement with family members and other adults (e.g., talking with parents, enjoying doing things with their family, helping at home; means = 3.60, 3.41, respectively; p < .05, effect size = .26).

There were no significant differences between program and comparison students in parental monitoring.

Prevention

Although program students generally scored lower than comparison students on measures of ATOD use, the only significant difference was for lifetime prevalence of marijuana use-fewer program (35%) than comparison students (46%) reported that they had ever used marijuana (p < .09, effect size = -.22).

There were no significant differences between program and comparison students in measures of perceptions about drugs use (i.e., negative attitudes toward drugs, perceived adult or peer disapproval of drug use, ability to resist peer pressure to use drugs, or drug use by best friend).

Program students were significantly less often in trouble at school than comparison students (means = 1.42, 1.59 respectively; p = .10, effect size = -.20). There were no significant differences in other measures of delinquency (i.e., frequency of fighting or damaging others' property, tendency toward aggressing/acting out, and number of times arrested/threatened with arrest in the past two months).

Youth Development

Program students reported that they participated in sports significantly more frequently than comparison students (means = 3.87, 3.30 respectively; p < .01, effect size = .39). There were no significant differences in other measures of frequency of activity participation (i.e., taking lessons/attending classes out of school, participation in clubs or youth groups, doing chores at home, or attending a summer program).

Program students scored significantly lower than comparison students on a measure of pessimism and negative self-appraisal (means = 1.83, 1.94, respectively; p < .10, effect size = -.20). However, there were no significant differences in the number of students who had been depressed for two or more weeks in the last two months. There were also no differences in students' belief in the value of perseverance, beliefs in conventional social values, and perceived social competence.

A Profile of the Evaluation of Say Yes First

Profile Information