You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.

www.HFRP.org

The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.

Terms of Use ▼

Program Description

Overview The Teen REACH (Responsibility, Education, Achievement, Caring, and Hope) program is an effort to provide positive youth activities during nonschool hours for Illinois youth. Teen REACH program providers are funded by the Illinois Department of Human Services (DHS). Funding goes to community-based agencies across Illinois such as local health departments, social service agencies, youth agencies, faith-based organizations, and community coalitions to implement the program. Teen REACH seeks to expand the range of choices and opportunities that enable, empower, and encourage youth to achieve positive growth and development, improve expectations and capacities for future success, and avoid or reduce negative risk-taking behavior. Its goals are to improve youth’s academic performance; provide opportunities for learning positive social skills, demonstrating positive social interactions, and building positive social relationships; encourage the adoption of positive decision-making skills that discourage harmful risk-taking behaviors; and strengthen parent-child bonds and community involvement.
Start Date October 1998
Scope state
Type after school, comprehensive services, weekend
Location urban, suburban, rural
Setting community-based organization, religious institution, recreation center, public school
Participants kindergarten through high school students (ages 6 through 17)
Number of Sites/Grantees 37 organizations with 73 program sites in 1998; 109 organizations with 250 program sites in 2004
Number Served over 50,000 from 1998 to 2004, approximately 30,000 annually
Components Programs are required to provide the following core services: (1) academic assistance, including homework time, basic skills tutoring, and enrichment programs to encourage creativity; (2) recreation, sports, cultural, and artistic activities to provide safe outlets for trying new skills and interests, building friendships and sense of belonging, and gaining developmentally relevant experiences; (3) positive adult mentors who maintain positive, sustained relationships with youth participants through one-on-one interactions; (4) life skills education that promotes abstinence from risky behaviors such as substance use, crime, violence, and sexual activity; and (5) parental involvement activities aimed at fostering parent/staff discussion and relationships, parent/child bonds, and community involvement. In addition, the Illinois DHS suggests that program participants should be given the opportunity to participate in at least one community service activity each year.
Funding Level $8.4 million in 1999; $18.4 million in 2000; over $19 million annually from 2001 to 2004
Funding Sources Illinois Department of Human Services (DHS)


Evaluation

Overview DHS’s Bureau of Community and Youth Services contracted with the Center for Prevention Research and Development at the University of Illinois to conduct an outcome evaluation for Teen REACH. During the pilot phase, the evaluation sought to understand youth outcomes at a small subset of program sites, in order to improve the overall evaluation process and provide data for continuous improvement. Second- and third-year evaluations have built on this pilot evaluation, adding a staff survey and including more programs in the evaluation sample. In the third year, work began on the development of program benchmarks and parent and teacher surveys that ultimately will be used to assess Teen REACH programs.
Evaluators Center for Prevention Research and Development at the University of Illinois
Evaluations Profiled Teen REACH: A Summary of the Pilot Evaluation

Teen REACH: Annual Evaluation Report 2002

Teen REACH: Annual Evaluation Report 2003
Evaluations Planned Teen REACH evaluations will continue to be conducted annually.
Report Availability University of Illinois, The Center for Prevention Research and Development. (2001). Teen REACH: A summary of the pilot evaluation. Champaign: Author. Available at www.cprd.uiuc.edu/trdocs.html.

University of Illinois, The Center for Prevention Research and Development. (2002). Teen REACH: Annual evaluation. Champaign: Author.

University of Illinois, The Center for Prevention Research and Development. (2004). Teen REACH: Annual evaluation report. Champaign: Author. Available at www.cprd.uiuc.edu/trdocs.html.


Contacts

Evaluation Kay Erwin Mulhall, M.Ed.
Coordinator of Research Programs
Institute of Government and Public Affairs
Center for Prevention Research and Development
510 Devonshire Drive
Champaign, IL 61820
Tel: 217-333-3231
Fax: 217-244-0214
Email: kemulhal@uiuc.edu
Program Karrie Rueter
Illinois Department of Human Services
Office of Prevention
535 W. Jefferson, 3rd Floor
Springfield, IL 62702-5058
Tel: 217-557-2943
Email: dhspab@dhs.state.il.us
Profile Updated December 1, 2004

Evaluation 3: Teen REACH: Annual Evaluation Report 2003


Evaluation Description

Evaluation Purpose To assess the relationship between program participation and youth educational and socio-behavioral outcomes, to develop local understanding and capacity for program sites to effectively participate in the evaluation, and to provide opportunities for grantees to understand, utilize, and continuously improve their programs through data-based decision making.
Evaluation Design Quasi-Experimental: The Illinois Department of Human Services (DHS) selected 30 program providers that had programs they judged to be well functioning and likely to cooperate with the evaluation process. The sample was selected to provide geographic representation from all regions in the state, include a range of types of provider agencies (e.g., community-based organizations, health departments), represent both rural and urban communities, assure cultural and racial diversity among participants, and include newly funded and more established programs. Each provider selected one program site to participate in the evaluation. These 30 sites included all 7 who participated in the pilot evaluation. Two of these 30 sites did not submit completed pretest and/or posttest youth surveys and thus were not included in the data analyses.

Evaluators used a single-group pretest/posttest design to assess changes in youth outcomes. Data were examined by comparing pretest and posttest data and by exploring the relationship between posttest data and program attendance levels or dosage (the total number of days youth attended). Pretest data were collected from 594 youth participants (out of 829 on program rosters for a response rate of 72%) and at posttest by 496 youth participants (out of 939 on program rosters for a response rate of 54%). The posttest response rate may be a low estimate because many youth on the program roster likely dropped out of the program by the time surveys were administered. A total of 332 youth completed both the pretest and posttest (the longitudinal sample). The analyses are based on either the longitudinal sample or, for some analyses, the full posttest sample.

Of the longitudinal youth sample, the average age was just under 13 years old, with 46% in Grades 4–6, 42% in Grades 7–9, and 13% in Grades 10–12. More than half were female (57%). The majority was African American (48%); the remainder were white (23%), Latino (20%), and other (11%). Eighty percent participated in the free/reduced-price lunch program at school. In terms of family composition at posttest, 57% were in two-parent families, 33% were in single-parent families, and 10% lived with non-parent adults (e.g., grandparents, foster parents, etc.).

Of participants who completed posttest surveys, 14% had been involved with Teen REACH for 3 or more years, an additional 14% for at least 2 years, 23% for at least 1 year, and 49% for less than 1 year, with an overall average of 1.03 years. At posttest, 54% of youth reported attending Teen REACH an average of 5 or more days a week, 20% for 4 days, and 26% for 1–3 days. The number of years in the program and the typical number of attendance days per week were used to create an estimate of total program dosage. This dosage is not just for the current program year, but takes into account the duration, in representing time since youth first started attending. Some analyses examined the relationship between dosage and change over time in the key outcome areas. Only the statistically significant correlations of dosage with outcomes are reported in the findings.

Staff surveys were collected in March 2003 from staff at 23 sites that were new to the statewide evaluation in the current year; the 7 sites continuing in the evaluation from the previous year were not asked to submit staff surveys again. Including the previous year’s data, surveys were collected from 135 staff at 29 of the 30 evaluation sites. (One site did not submit surveys.) Based on the number of staff sampled by each site, this represents a response rate of approximately 49%. Of staff who answered a question about their role within Teen REACH (n = 130), 53% were direct care staff, 18% were site coordinators, 16% were project directors, 2% were volunteers, and 11% classified themselves as “other.”
Data Collection Methods Surveys/Questionnaires: Youth in Grades 4 and above at the selected sites were surveyed in October/November 2002 (pretest) and again in April through June 2003 (posttest). The survey instrument included items and scales to assess: youth background/demographic information, program exposure/dosage, perceived connections with staff, perceived program impacts (the frequency with which specific topics are addressed in Teen REACH and perceptions of program benefits), and youth outcomes. Youth outcome areas included frequency of homework completion, school attendance (days absent in the past month, and reasons for these absences), academic performance (self-reported grades), academic aspirations (e.g., “how important is it to you to graduate from high school?”), quality of school life (e.g., frequency with which youth are happy when they are in school), engagement in prosocial and delinquent behaviors (e.g., “in the last 6 months, how often did you tell lies or cheat?”), substance use (reports of alcohol, tobacco, and drug use during the past 30 days), leadership (e.g., “I like to set a good example for other young people”), self-concept (e.g., “I am happy with myself as a person”), adult connections (e.g., “I know people/places in my community where I can get help with a personal problem”), peer-group cohesion (e.g., “I enjoy the time I spend with my friends”), problem-solving skills (e.g., “when I face a new problem, I first try to find out what it is”), and parent involvement in child’s education and Teen REACH (e.g., the frequency with which parents visit the program or their school).

Administered in March 2003, the staff survey assessed staff characteristics, roles, and responsibilities; organizational issues; programming focus; and program impacts on youth. To assess program focus, staff were asked to rate potential areas in which their Teen REACH programs tended to focus: academics, youth development, sports/recreation, and parent involvement/family programming. Each area was rated on a 4-point scale from not at all a focus to a major focus of their program. These data were aggregated at the site level to produce an indicator of how much the site focused on that area. This information was linked to youth survey data at the same site to assess the relationship between program focus and youth outcomes.
Data Collection Timeframe Data were collected during the 2002–2003 school year.


Findings:
Formative/Process Findings

Activity Implementation Program focus areas that staff rated the highest were academics (providing homework help, stressing the importance of education, tutoring in certain subjects) and life skills (getting along with peers, conflict resolution, general problem solving). Also rated highly were recreational activities and opportunities to establish positive caring relationships with adults. Parent- and family-focused programming areas were not rated as highly as a program focus.

At programs with a stronger academic focus, youth reported significantly higher posttest survey ratings on how much they learned academically, feelings about school, and academic aspirations (p < .05 for each) than youth in programs with less of an academic focus. No significant differences were found for self-reported grades or homework completion.

Youth in programs with a stronger youth development focus reported significantly higher posttest survey ratings on how much they learned in this area (p < .05) than youth in programs with less of a focus in this area. There were no significant differences for ratings of other youth development-related outcomes (self-concept, leadership skills, problem-solving skills, peer-group cohesion, adult connections).

No significant differences were found between programs with a high or low sports/recreation focus on any of the areas assessed on the posttest youth survey that evaluators felt might affect sports/recreation-related outcomes (self-concept, leadership skills, problem-solving skills, peer-group cohesion).

Youth in programs with a stronger parent involvement/family focus reported significantly higher posttest survey ratings on parent involvement in youth’s education (p < .05) than youth in programs with less of a focus in this area.

Of surveyed staff, 37% said that most youth and families received the services they needed.


Summative/Outcome Findings

Academic Significant increases were found from pretest to posttest in ninth to twelfth grade participants’ academic aspirations and the quality of school life (p < .05 for each). No significant differences were found in any academic outcome areas for fourth to eighth grade participants.

Over half of participants’ outcomes increased or stayed the same from pretest to posttest in self-reported grades (71%), academic aspirations (72%), and school life quality (51%).

For fourth to eighth graders, higher dosage levels were significantly correlated with increases over time in academic aspirations (p < .01) and positive feelings about school (p < .05).

For the fourth to eighth graders in their first year of Teen REACH, higher dosage levels were significantly correlated with more positive changes over time in homework completion and academic aspirations (p < .01 for each).

For ninth to twelfth graders in their first year of Teen REACH, higher dosage levels were negatively correlated with homework completion (p < .05).

When asked about program benefits, a majority of youth reported that they learned a lot about completing homework (62%), getting help with certain subjects in school (52%), and studying for tests (51%). Similarly, some youth provided open-ended survey responses that indicated that the program helped them with their homework and to get better grades.

A number of the surveyed staff reported that most participants had the following outcomes: completed homework more often (58%), had better school attendance (44%), did better in school/made better grades (37%), and improved their attitudes toward school (27%).

In response to an open-ended question about program impacts, many staff reported improved academic performance (e.g., “when I started with him, he was not reading chapter books, and now five months later, [he] can”).
Family For the total sample of youth, 51% of parents maintained or improved their levels of school involvement from pretest to posttest. Significant decreases were found from pretest to posttest in fourth to eighth grade participants’ parent involvement in education (p < .01). The differences for ninth to twelfth graders in this area were not significant.

For ninth to twelfth grade participants in Teen REACH for the first time, higher dosage levels were negatively correlated with parent involvement (p < .10).

Twenty-five percent of surveyed staff indicated that most youth involved in the program increased their positive interactions with their parents.

Of surveyed staff, 22% reported that most parents involved in the program knew more about the resources available in the community, and 21% indicated that most of these parents were more involved in their child’s education and knew more about how to help their child do well in school.
Prevention For fourth to eighth graders and ninth to twelfth graders in their first year of Teen REACH, higher dosage levels were significantly correlated with reduced negative/delinquent behavior (p < .05 for fourth to eighth graders and p < .10 for ninth to twelfth graders).

When asked about program benefits, a small majority of youth reported that they learned a lot about the dangers of drugs (53%) and the skills needed to avoid drugs (51%).

Of the staff surveyed, 36% reported that most participants were less likely to become involved with drugs, engaged in less negative behavior/fewer discipline problems, and had stronger anti-drug attitudes. In addition 33% of staff indicated that most youth decreased their drug use, and 16% reported that most youth increased their knowledge of reproductive and sexual health issues.

In response to an open-ended survey question asking about youth’s program experiences, some youth said that it gave them somewhere to go, or something to do (e.g., “it keeps me off the streets and out of trouble”).
Youth Development Significant increases were found from pretest to posttest for ninth to twelfth grade participants in peer group cohesion and adult connections (p < .05 for each). No other significant differences in youth development outcomes were found for ninth to twelfth grade participants.

Significant increases were found from pretest to posttest in fourth to eighth grade participants’ delinquent behavior (p < .01). No other significant differences in youth development outcomes were found for fourth to eighth grade participants.

From pretest to posttest, a number of participants’ outcomes either improved or stayed the same, including adult connections (59%), self-concept (55%), peer-group cohesion (55%), leadership (53%), and negative/delinquent behavior (47%).

For participants in Grades 9–12, a significant positive correlation was found between program dosage and the development of leadership skills (p < .01).

For fourth to eighth graders in their 1st year of Teen REACH, higher dosage levels were significantly correlated with more positive changes over time in the areas of problem solving and peer-group cohesion (p < .10 for each).

For ninth to twelfth grade participants in Teen REACH for the first time, higher dosage levels were negatively correlated with self-concept (p < .05) and adult connections (p < .10).

In response to an open-ended survey question about youth’s program experiences, some youth indicated that the program helped them further their education/plans for the future/goals (e.g., “Teen REACH has helped me see that I am highly intelligent and can be something in life”). Some youth also indicated that the program helped improve their relationships with peers and friends. In addition, some youth indicated that the program exposed them to new activities (e.g., “I have a good time at trying other things”).

Of surveyed staff, the following percentages reported that most program youth had positive outcomes: exposure to new activities/places/events (61%), increased happiness (56%), increased prosocial/recreational interests (51%), established positive relationships with caring adults (44%), more self-confidence/improved self-esteem (42%), getting along better with others (38%), improved physical/sports skills (37%), improved problem-solving and decision-making skills (35%), increased leadership skills (33%), improved health-related habits (exercise, hygiene; 28%), and set goals for the future (19%).

When asked about program benefits, a number of youth reported that they learned a lot about the following areas: setting goals for the future (51%), solving problems and making good decisions (51%), getting along with others (48%), preparing for a future job or career (46%), how to be a better leader (45%), dealing with conflicts with others (42%), the importance of exercise and hygiene (40%), and learning about other cultures (30%).

In response to an open-ended question about program impacts, many staff focused on improved participant attitudes and behavior, e.g., “There is one young man that I have observed who experiences an amazing transformation. A couple of years ago I considered him aloof and bordering on disrespectful. Today he is in a position of leadership teaching younger children. He is positive and polite.” Some staff also commented on youth’s improved self-concept, e.g., “There was a child who starting Teen REACH was very shy. He now is able to open up, share his thoughts and ask others to play with him. His attitude towards others is happy and welcoming.”

© 2016 Presidents and Fellows of Harvard College
Published by Harvard Family Research Project