Search Publications | The Evaluation Exchange | FINE Network | OST Database

The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.

A Profile of the Evaluation of Thunderbirds Teen Center Program

Program Description

Overview	The Thunderbirds Teen Center Program is a multi-functional facility in North Phoenix, Arizona, operated through the City of Phoenix Parks and Recreation Department. The Teen Center's mission is to promote the positive self-development of teens, ages 13 to 19, by providing a comprehensive service system during out-of-school time that focuses on the whole individual. The primary goal of the Teen Center program is to provide students with an inventory of skills and positive experiences that will enable them to be more successful at school, and ultimately improve the chances that these students will remain in school. The Teen Center also hopes to have a long-term impact on reducing the incidence of juvenile delinquency within the surrounding community.
Start Date	1996
Scope	local

Type	after school, summer/vacation, comprehensive services, outreach programs at middle and high schools
Location	urban
Setting	recreation center located five blocks from closest high school (approximately 70% of center participants) and six blocks from two closest middle schools (approximately 20% of center participants)
Participants	middle school and high school students
Number of Sites/Grantees	1 center and outreach programs that reached over 15 schools per year
Number Served	Over 500 teens visited the center during the study period (1997). Approximately 40% of the teens who visited the center participated in structured programs (vs. drop-in). Outreach programs reach at least an additional 2,000 youth each year.
Components	Opportunities at the Teen Center are available to teens ages 13 to 19 and include the following structured programs: LIFE (Hip Hop Dance): This dance program has been offered since 1995 with the goals of providing youth with positive alternative recreation, increasing self-esteem, aiding in developing positive peer and staff relationships, enhancing communicative skills, and redirecting negative energy into positive behavior. Basketball: The goals of this program are to build teamwork, self-esteem, communication skills, and positive self-image through recreation and peer interaction. City Streets: This nationally recognized mobile urban program began in 1986 and is now the name attached to several smaller programs. The program's purpose is to provide Life Management Leadership Development to teens by providing information/training in decision-making skills designed to: 1) develop skills conducive to supporting gang drop-outs and preventing school drop-outs; 2) develop self-discipline, social, and self-preservation skills; and 3) build character and cultural awareness. Recreation Intern Program (RIP): The goals of this decade-old summer volunteer program are to provide youth ages 15 to 19 with the opportunity to gain practical experience in the field of recreation, receive educational training to support this experience, and increase present job skills, knowledge, and self-esteem. Funded by the U.S. Department of Labor, the program pays students a minimum wage 10 hours per week. In 1997, this program began to be offered year round with youth being mentored by Parks and Recreation Department staff, attending workshops and gaining paid work experience with Parks and Recreation Department programs. Project BRAVE: This program began in 1997. Youth Counselors provide workshops and presentations to over 20 sites in the Northwest region of the city (one of three regions). These sites have included schools, after school programs, and special events. Sites can choose from 11 workshops that focus on three goals: 1) to increase education and awareness concerning violence in our community, 2) to provide program participants with positive alternatives to violent behavior, and 3) to assist youth in self-empowerment. B-Boys/B-Girls: The boys' program has been offered since 1996 and the girls' program has been offered since 1997. The goals of these breakdancing programs are to provide youth with positive alternative recreation, enhance communication skills, redirect negative energy into positive behavior, and educate teens about the art of breakdancing and about Hip Hop culture. Teen Council: This program began in March 1996 as part of the National City Streets program. The goal of the program is to form a core group of teens to build communications skills and cooperation through educational, recreational, and community programs. TechCare: This integrated staff and peer interaction program began in May 1996. The program gives teens access to multimedia tools including computers, a digital recording studio, and graphics/arts components. In addition, the program provides tutoring and term paper assistance and other activities designed to provide positive alternatives and life skills assistance. North Canyon Suspension Program: Begun in Spring 1996, this program provides drug intervention/education for students ages 14 to 17 who have been arrested and suspended for the use of drugs and/or alcohol in Paradise Valley High Schools. The program aims to reduce the number of days of suspension from nine to five and the likelihood of failure or further infraction. Teens must complete a one-hour interview plus eight hours of the program. Outdoor Adventure Trips: These trips are offered in the summer near Flagstaff, Arizona, and include hiking, caving, outdoor living, and environmental education. The goals of the trips are to increase awareness of the outdoors and to facilitate teamwork, communication, and problem-solving skills.
Funding Level	In 1997 the annual operating budget of Thunderbirds Teen Center was $150,000. In 2002 the annual operating budget is $180,000. Approximately 10% of the annual operating budget is obtained from grant funding.
Funding Sources	COMCARE (a state funded behavioral agency in Arizona which divides moneys into different programs all over the state) and the City of Phoenix Parks and Recreation Department. The At Risk Youth division has also received funding from a $0.25 surcharge at each of the cities four golf courses (fifth course was added in 2000).

Evaluation

Overview	The evaluation examined the effects of program participation on youth in a teen center. More specifically, the researchers were interested in exploring the effects of an organized recreation teen center setting on youths' self-esteem, academic performance, resiliency, and perceived family well-being. The evaluation was conducted during the period of January 1997 to March 1998.
Evaluator	Dwayne Baker, Michigan State University; John Hultsman, Arizona State University-West; Barry Garst, Research Assistant
Evaluations Profiled	Thunderbirds Teen Center Program Evaluation
Evaluations Planned	An evaluation of Project BRAVE that now operates citywide and is still based out of Thunderbirds Teen Center is currently under final review and will be available on Dr. Baker's website at: www.prr.msu.edu/baker/research.htm.
Report Availability	Baker, D., Hultsman, J., & Garst, B. (1998, March). Thunderbirds Teen Center Program evaluation. [Available on Michigan State University website: www.prr.msu.edu/baker/ThunderBirdReport.pdf (Acrobat file) and Texas A&M University website: rptsweb.tamu.edu/Faculty/Witt/conpubs/thunder.pdf (Acrobat file).]

Contacts

Evaluation	Dwayne Baker, Ph.D. Park, Recreation and Tourism Resources Michigan State University RM-131 Natural Resources Building East Lansing, MI 48824-1222 Tel: 517-353-5190 ext. 115 Email: dbaker@msu.edu Website: www.prr.msu.edu/baker/main.htm	John Hultsman, Re.D. Arizona State University West PO Box 37100 Phoenix, AZ 85069-7100 Tel: 602-543-5500 Email: jhultsman@asu.edu
Program	Manny Tarango, Division Head At Risk Youth Division City of Phoenix Park, Recreation and Library Department 2705 North 15th Avenue Phoenix, AZ 85007 Tel: 602-262-7370 Fax: 602-262-7333 Email: manny.tarango@phoenix.gov	Tim Valencia, M.C. Center Director The Thunderbirds Teen Center 1104 East Grovers, Building A Phoenix, AZ 85022 Tel: 602-495-7323 Fax: 602-534-4515 Email: tim.valencia@phoenix.gov
Profile Updated	July 14, 2002

Evaluation: Thunderbirds Teen Center Program Evaluation

Evaluation Description

Evaluation Purpose	To examine the following questions: Did participation in Thunderbirds Teen Center recreation programs influence participants' perceptions of self, family, and their community and lead to increased self-esteem and increased perceptions of risk-protective factors? Did participation in the Thunderbirds Teen Center after school recreation program influence school performance, e.g., decreased number of absences and tardies and increased school grades?
Evaluation Design	Quasi-experimental: The study was conducted in two phases. In Phase 1, pre-test/post-test assessments (using standardized instruments) were administered to three participant groups during three sessions (spring, summer, and fall): 1) non-participants (n=180), a comparison group of student cohorts from local middle and high schools; 2) new participants (n=191), who visited the Teen Center for the first time during the study period; and 3) regular participants in structured programs (n=81). Forty-nine percent (48.9%) of the sampled youth returned a parental permission form, as well as completed both pretest and posttest assessments Phase 2 consisted of interviews conducted with three participant groups and one nonparticipant group during the fall of 1997. Teen Center intake data were analyzed to identify three groups of Teen Center participants: 1) drop-out teens who discontinued Teen Center participation, 2) infrequent teens who participated in Teen Center programs/drop-in irregularly, and 3) frequent teens who regularly participated in Teen Center programs/drop-in. A fourth group consisted of nonparticipants from each of four grades (7th, 8th, 9th, and 10th) from local high and middle schools. Teens from each of these groups were randomly selected and given a parental/teen consent form for participation in an interview. A total of 49 teens were identified for possible participation in the interviews. Out of this group, 19 teens agreed to be interviewed and completed the parental/teen consent forms: three drop-out participants, three infrequent participants, six frequent participants, and seven nonparticipants. In addition, demographic data were collected.
Data Collection Methods	Interviews/Focus Groups: Interviews were conducted to identify: after school program activity participation (i.e., doing homework at home, hanging out with friends, engaging in school and nonschool sports, working, going to the Teen Center, etc.); perceptions of Teen Center; reasons for participation frequency or infrequency; ways to improve community awareness of, and participation in, Teen Center program; and Teen Center/recreation participation benefits. Interviews were conducted in Spring 1998. Secondary Sources/Data Review: Demographic data on participants (Spring 1997 - Fall 1997) were collected through assessments, intake records, and school records. School data included information on grades, attendance, and discipline. Tests/Assessments: Pre-test and post-test assessment instrumentation included: Risk Protective Factors Scale (Witt, Baker & Scott, 1996); exploring participants' risk protective factors (neighborhood resources, knowledge of safe places to play, sense of belonging, caring adults, models for conventional behavior, value on achievement, controls against deviant behavior, and liking/perceived competence); Harter's (1988) Self-Perception Profile for Adolescents, looking at aspects of self-esteem (on five subscales: global self-worth, scholastic competence, job competence, social acceptance, and behavioral conduct); Life Events Checklist (adapted from Pryor-Brown & Cohen, 1989), identifying significant stressful life events in teen lives over two time periods (12 months prior to the test and the period of time between pre-test and post-test); and F.A.C.E.S. II (Olson, Bell & Porter, 1985), measuring family functioning (family adaptability and cohesiveness). Assessments were administered in Spring 1997, Summer 1997, and Fall 1997.
Data Collection Timeframe	Data were collected Spring 1997 - Spring 1998.

Findings:
Formative/Process Findings

Activity Implementation	Nonparticipants' desired activities in out-of-school time included sports related activities such as basketball, baseball, and pool and ping pong tournaments. Both nonparticipants and drop-out participants suggested that the Teen Center could provide activities that are not available at school because they were discontinued, offered only once a year, or not offered at all. They expressed the need for fun, exciting programs to attract new participants, schoolwork assistance programs, and programs that target junior and senior high school seniors, such as job-assistance and housing assistance programs. Teen Center staff function as behavioral role models for teens in structured programs.
Program Context/Infrastructure	Teen Center staff who were involved in the evaluation process felt that the Teen Center should continue to monitor program outcomes and quality through regular evaluation. The staff demonstrated a willingness to support evaluation, and the ability of staff to assess the impact of their programs was enhanced by this evaluation process. The staff's commitment to program evaluation may have contributed to the increased numbers of Fall 1997 parental/teen consent forms returned, as they made great efforts to encourage participation and monitor the forms. Interview data suggested that Teen Center participants (infrequent and frequent) were more likely than nonparticipants to perceive North Phoenix and the Teen Center areas as a safe place to be. Nonparticipants were less able to identify a safe place.
Recruitment/Participation	Interview data revealed that many teens are unaware of Teen Center programs. Teens suggested a multi-dimensional approach to advertising and promotion. Over all participant categories, the four most effective ways for advertising and promoting the Teen Center programs included: flyers, radio, school announcements, and school visits by Teen Center staff and teens to tell students about Teen Center opportunities. Interview data suggested that some teens feel that Teen Center programs are concentrated on Latino teens. This perception may have reflected Teen Center staff's efforts to provide services to an important segment of their target group. New participants and regular participants experienced more stressful life events 12 months prior to the study than nonparticipants. New participant and regular participants' stressful life events decreased from pre-test to post-test, while nonparticipants rates of stressful life events stayed about the same. Twelve months prior to the assessment, two stressful life events were most common to all participant categories: 1) “someone I know has been injured as a result of violence by someone else” and 2) “I got a bad mark on a test.” New participants were the only group to identify “I was sent to the principal” as one of the top three stressful life events. Nonparticipants and regular participants identified “someone threatened me” as one of the top three stressful life events. In each participant category, the two least frequently occurring stressful life events were “I'm having a child” and “one of my parents died.” In addition, few nonparticipants identified “suspended from school;” few new participants identified “new baby sibling;” few regular participants identified “new baby sibling,” and “parents remarried.” In the period of time between pre-test and post-test, two stressful life events were most common to all participant categories: 1) “someone I know has been injured as a result of violence by someone else” and 2) “I got a bad mark on a test.” Nonparticipants were the only group to identify “parents worked a lot and were not home very much” as one of the top three stressful life events. The three stressful life event categories identified least often in each participant category were: (1) nonparticipants—“I'm having a child,” “one of my parents died,” and “I was injured as a result of others’ violence;”( 2) new participants—“one of my parents lost their job,” “I'm having a child,” and “one of my parents died;” and (3) regular participants—“I'm having a child,” “one of my parents had a serious accident,” and “I was injured from others’ violence.” Interview data identified other stressful life events and important “issues” with which the teens were faced: schoolwork, drugs and alcohol, death of a close friend, and moving out of a parents' home/finding a job. Nonparticipants had no previous experience at the Teen Center. In addition, most had never heard of the Teen Center. Drop-out participants and infrequent participants had heard very little about the Teen Center from their school peers. Frequent participants were much more likely to have heard peers talking about their Teen Center perceptions. Some of them were positive. Drop-out, infrequent, and frequent participants had heard peers call the Teen Center “boring.” All participant groups, with the exception of nonparticipants, tended to have six months or more experience with the Teen Center. Most said that they had begun to attend the Teen Center soon after it opened. Drop-out and infrequent participants did not attend Teen Center structured programs regularly, although they did drop-in occasionally. Frequent participants tended to participate in at least one program regularly, attend a second or third program once in a while, and drop-in. Frequent participants usually switched programs after participating for one semester. Interviews revealed that many nonparticipants were unaware of Teen Center opportunities. Drop-out participants cited school sports and an inability to identify with Teen Center participants as reasons they stopped attending the Teen Center. Infrequent participants discussed increased school work loads and job responsibilities as contributing to their Teen Center participation rates. In general, the two most often stated reasons that more teens do not get involved in Teen Center programs were: 1) Teen Center is boring/would rather do other things and 2) would rather be drinking or doing drugs (especially smoking marijuana). Another reason for nonparticipation was that nonparticipants perceived a differences in participants' social status and ethnicity compared to their own. Nonparticipants' measures of self-esteem (Scholastic Aptitude Test, Behavioral Conduct, Social Acceptance, Global Self-Worth) were higher than those of the other two participant groups. However, regular program participants scored significantly higher than new participant groups on Harter's Self-Perception Profile for Adolescents scales overall and Behavioral Conduct. Life events, self-perception scores, and protective factor scores suggest that the Teen Center recruited participants who felt they were most at risk of engaging in delinquent behaviors. However, Teen Center programs and facilities may not be meeting new participants' perceptions, leading to boredom and disinterest. This could account for the large proportion of teens who dropped out of the Teen Center Program after one or two visits.
Satisfaction	Assessment respondents overall rated the Teen Center facilities and staff as above average. Nonparticipants did not rate the Teen Center as highly as new participants and regular participants. The two areas which were rated highest at the posttest measure by teens familiar with the teen center were: “respect that Teen Center shows teens” and “Teen Center staff overall.” For Teen Center new participants and regular participants, ratings from pre-test to post-test tended to increase or stay the same, with the exception of Teen Centers overall. New participants gave facilities a lower rating on the post-test measure than they did on the pre-test measure. This suggests that new participants' expectations of Teen Center facilities were not met. Regular participants in structured programs are enjoying Teen Center recreation programs more than new participants.

Summative/Outcome Findings

Academic	Results suggest some school-related benefits of Teen Center participation, because 1) grades decreased less in regular participants compared to new participants, 2) tardy rates for new participants were lower compared to those of nonparticipants, and 3) some teens take advantage of academic assistance from Teen Center staff. Among all teens, there were no significant overall decreases in rates of school truancy, excused absences, or tardies. All three rates increased from pre-test to post-test, with the exception of nonparticipant school truancy rates. Among teens who completed the pre-test and post-test, new participants tardy rates were lower than nonparticipants' tardy rates. Among all teens, nonparticipants' rates of total disciplinary action taken decreased from pre-test to post-test, and new participants' rates increased from pre-test to post-test. The grades of nonparticipants and new participants decreased during the time period from pre-test to post-test. However, there was no significant difference in nonparticipants and new participants' scores in any of the subjects that were examined (English, Math, Science, and Social Studies). Among teens who completed pre-test and post-test, new participants grades decreased to a lesser degree than nonparticipants' grades. Although post-test school data (grades, attendance, discipline) were not collected for regular participants, interview data suggested that frequent participants' academic performance increased because the Teen Center gave them a quiet place to complete homework, and Teen Center staff served as a resource when teens needed assistance. Teen Center participants' school academic performance, attendance, and discipline were very poor. However, results suggest that for all participant groups the post-test period may have been more academically challenging. This could be due to increases in schoolwork or schoolwork level of difficulty. Both new participants and regular participants had a higher rate of stressful life events than nonparticipants. Evaluators conjecture that these stressful life events could have impacted school performance. Therefore, without the Teen Center, participants may have seen a more detrimental effect on academic achievement. Nonparticipant and new participants rates of school enrollment in arts, music, and trades-related courses decreased from pre-test to post-test. In general, new participants were more likely to take art and music courses in school than nonparticipants. Nonparticipants were more likely to take trade-related courses in school than new participants.
Community Development	For regular participants, Teen Center is functioning as a link between teen and community. However, a portion of the teen population in North Phoenix are unaware of Teen Center opportunities.
Youth Development	Regular participants in structured programs are learning more about appropriate forms of behavior than participants who are new and may not be as highly involved. Interview data suggest that Teen Center participation increased participants’ opportunities for: positive peer interaction, meeting new friends, and developing a tolerance for individual differences. Interview data suggested that frequent (regular) participants experienced self-esteem increases, which were represented by positive changes in behavior. Evaluators believe that teens' resiliency improved resulting from participants' ability to recognize the Teen Center as a safe place to be and as a link between youth and community. Moreover, Teen Center program participation increased participants' knowledge of neighborhood resources. Interview data suggested that Teen Center staff made a positive impact on Teen Center frequent and infrequent participants. Regular participants in structured programs are becoming more competent in recreation activities.

A Profile of the Evaluation of Thunderbirds Teen Center Program

Profile Information