Search Publications | The Evaluation Exchange | FINE Network | OST Database

The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.

A Profile of the Evaluation of Fort Worth After School Program

Program Description

Overview	Fort Worth After School (FWAS) program is a community-driven initiative that provides after school programs for youth in Fort Worth, Texas. Program goals revolve around three areas: educational competence, physical and social development, and crime reduction. Services provided include homework assistance, tutorial, snacks, cultural and recreational activities, and mentoring.
Start Date	September 2000
Scope	local
Type	after school
Location	urban
Setting	public school

Participants	preschool through middle school students
Number of Sites/Grantees	52 schools (47 elementary schools and 5 middle schools, 2000–2004); 55 schools (48 elementary schools and 7 middle schools, 2004–2005)
Number Served	In the first 5 years, 16,252 youth attended at least 1 year of FWAS. In year 5, funding was allocated to support 50–150 youth per day at each campus.
Components	A Coordinating Board (CB) made up of city, school district, and community representatives oversees the program, while an administrative staff monitors day-to-day operations. Most programs operate 4 days per week (Monday–Thursday) from the time school is dismissed until 6 p.m. At many of the sites, the same youth participate all 4 days each week. At several sites, some youth participate 2 days per week, with a different group of youth participating the other 2 days. In a few cases, programs are offered on Fridays. At each site, youth receive a snack and an opportunity to complete homework. Youth receive a combination of enrichment activities and free play or recreation time. Service providers include programming in at least three of five areas: academic enrichment, sports and recreation, cultural and fine arts, community service, and character development. FWAS established a youth-to-staff ratio minimum for all sites: 15:1 for elementary schools and 20:1 for the middle school. The CB's selection of school sites was based on a combination of factors including the percentage of youth eligible for the Federal Free Lunch Program and crime statistics for the area around the school. Each year the CB approves several agencies/organizations as potential service providers. Principals at participating schools either select one of these providers to offer the program on their campus or develop a “site-based” (self-managed) program. FWAS is staffed by a director and two program coordinators. A Parent Advisory Committee (PAC), formed in year 2, is made up of parents of program participants. In addition, in year 3, monthly networking meetings hosted by service providers, schools, and community organizations were established to give site coordinators, providers, and principals the opportunity to talk, share, and collaborate.
Funding Level	The city of Fort Worth and Fort Worth Independent School District (FWISD) jointly provided a total of $2.2 million per year for the first 5 program years. Some sites used additional funding sources to serve more youth; some are supported in conjunction with FWISD 21st Century Community Centers (21st CCLC) programs.
Funding Sources	City of Fort Worth Crime Control and Prevention District, FWISD, U.S. Department of Education 21st Century Community Learning Center funds

Evaluation

Overview	The evaluation sought data about program inputs (the elements that make up FWAS, such as its leaders, facilities, and activities); outputs (the number of youth attending FWAS, the types of activities offered, and the number of program hours provided); and outcomes (what happens to youth, parents, and the community as a result of FWAS).
Evaluators	Peter A. Witt, Terri King, Jin-Hyung Lee, Kristi Montadon, Lydia Justice, Joanne Oh, Billy Brown, Texas A&M University
Evaluations Profiled	Fort Worth After School Program: “A Diamond in the Rough” First Year Evaluation Fort Worth After School Second Year Evaluation Fort Worth After School Third Year Evaluation Fifth Year Evaluation: Fort Worth After School
Evaluations Planned	Evaluations are ongoing.
Report Availability	Witt, P. A., & King, T. (2001). Fort Worth After School program: “A diamond in the rough” first year evaluation. College Station, TX: Texas A&M University. Available at rptsweb.tamu.edu/Faculty/Witt/1Report%20Text.PDF Witt, P. A., King, T., & Lee, J. H. (2002). Fort Worth After School second year evaluation. College Station: Texas A&M University. Available at rptsweb.tamu.edu/Faculty/Witt/ FortWorth2003/FWASEVALS.htm Witt, P. A., King, T., & Montandoni, K. (2003). Fort Worth After School third year evaluation. College Station: Texas A&M University. Available at: rptsweb.tamu.edu/Faculty/Witt/ FortWorth2003/FWASEVALS.htm Witt, P. A., King T., Justice, L., Oh, J., & Brown B. (2005). Fifth year evaluation: Fort Worth After School. College Station: Texas A&M University. Available at: rptsweb.tamu.edu/ Faculty/Witt/FortWorth2003/FWASEVALS.htm

Contacts

Evaluation	Peter A. Witt Elda K. Bradberry Recreation and Youth Development Chair Department of Recreation, Park, and Tourism Texas A&M University 2261 TAMU College Station, TX 77843-2262 Tel: 979-845-7325 Email: pwitt@rpts.tamu.edu
Program	Miguel Garcia Director Fort Worth After School 100 N. University NE 232 Fort Worth, TX 76107 Tel: 817-871-2369 Fax: 817-871-2893
Profile Updated	May 24, 2006

Evaluation 4: Fifth Year Evaluation: Fort Worth After School

Evaluation Description

Evaluation Purpose	To determine which elements are necessary to offer a quality program, and what benefits for youth are achieved through participating in the program.
Evaluation Design	Non-Experimental: Evaluators assessed stakeholders' program perceptions and satisfaction; FWAS attendance; and participants' social behavior, school liking, homework completion, and other academic abilities. Surveys were collected from 991 parents at 47 sites, 1,693 youth at 48 sites, 51 principals, 40 site coordinators, and 191 program staff at 45 sites. Program attendance, academic, school attendance, and other data were collected on all participants over the first 5 years to explore the impact of FWAS attendance on academic performance and school attendance. Site visits and program staff interviews were conducted at all sites.
Data Collection Methods	Document Review: FWAS staff provided data on program goals, activities, cost, and attendance. Interviews/Focus Groups: Interviews and focus groups discussed program quality, outcomes, satisfaction, goals, and activities. Observation: Site visits were made to all sites at least 3 times during the school year to assess program quality. Secondary Source/Data Review: Participants' school attendance and test-score data were collected from schools. In addition, program registration, participant demographic, and daily attendance data were collected from service providers. Surveys/Questionnaires: Surveys asked about program goals, activities, quality, satisfaction, and outcomes. Tests/Assessments: Scores on the Texas Assessment of Knowledge and Skills (TAKS™), the statewide assessment program, were collected for FWAS participants.
Data Collection Timeframe	Data were collected during the 2004–2005 program year.

Findings:
Formative/Process Findings

Costs/Revenues	The cost per hour declined from $2.62 per child per hour in year 1 to $1.67 in year 5. In year 5, FWAS benefited from in-kind donations valued at $102,860 from 37 sources and 1,500 volunteer hours from 53 organizations.
Program Context/Infrastructure	When surveyed, 38% of youth agreed that they were afraid when walking in their neighborhoods. Youth in grades 3–5 reported more fear (38%) than those in grades 6–8 (30%). In contrast, 92% of youth agreed that they felt safe in FWAS. Parents indicated on surveys that if their children were not in FWAS, 62% would have had adult supervision almost all of the time, with another 18% having adult supervision some of the time. In some cases, parents indicated that to be supervised, youth would have had to go to work with them or that supervision might have been provided by a sleeping adult. Without FWAS, 20% of parents felt that their child would have lacked adult supervision after school. Of youth surveyed, 49% agreed that there was nothing to do during after school hours besides FWAS, and 75% of parents agreed that if not in FWAS, their child would have had little to do.
Program–School Linkages	From years 1–5, the number of schools managing their own programs increased from 3 to 22 (the number of external providers decreased from 10 to 5). This change was due to several factors. For example, principals felt that if programs were self-managed, they could better align FWAS with school goals; teachers would be well-positioned to know the youth and their needs; and they could allow the use of more school facilities. However, the number of site-based programs leveled off by year 5, as external providers made adjustments to improve quality and work with schools. As a result, principals indicated greater satisfaction with their providers in year 5, and few additional changes were made. Teacher involvement in FWAS usually led to greater confidence by principals and school staff that school facilities would not be abused. Consequently, some external providers hired teachers as site supervisors or program staff to ensure stronger ties to the school and greater confidence in program staff's ability to care for school facilities.
Recruitment/Participation	Of 5th year registrants, 51% were girls and 49% were boys. Twenty-seven percent were in grades Pre-K–2, 52% were in grades 3–5, and 22% were in grades 6–8. Fifty-nine percent were Hispanic, 34% were African American, 6% were White, and 2% were other ethnicities. In addition, 30% were classified as Limited English Proficient; 92% were eligible for the Federal Free Lunch Program; and 56% were considered at-risk of school failure, based on Forth Worth Independent School District criteria. In year 5, there were 3,695 FWAS attendance slots with 6,364 youth enrolled at sometime during the year, with a total of over 1.4 million child hours of attendance. FWAS operated an average of 132 days across sites during year 5. Since the program's inception, 16,252 different youth attended at least 1 year of FWAS, with 67% attending only 1 year, 23% enrolled for 2; 8% enrolled for 3, and 3% enrolled for 4-5. Approximately 50% of year 5 participants attended 2 or more years. FWAS operated at over 100% of projected daily capacity in years 3–5 (105% in year 5). Youth participated in the majority of days enrolled (92% in year 5). As youth moved from elementary to middle school, the average number of days attended in a given year decreased.
Satisfaction	Of youth surveyed, 72% agreed that they liked FWAS, 73% indicated that they felt good about themselves when at FWAS, and only 14% indicated that they felt bored during FWAS. Over 95% of surveyed parents reported that they would recommend FWAS to others, and 93% would sign their child up again. Some parents who said they wouldn't sign their child up noted that their child would be moving on to middle or high school. Youth in grades 3–4 were more likely to say that they wanted to sign up next year than those in grades 6–7 (63% vs. 44%), with youth in grades 5 and 8 even less committed (39% and 30%). The most common responses to what youth liked best about FWAS were fun games and activities, improving social skills, sports, playing outside (grades 3–5), teachers/staff (grades 3–5), technology classes (grades 6–8), and variety of activities (grades 6–8). The most common responses to what youth would change were snacks, nothing, more fun activities and classes, and less discipline.
Staffing/Training	According to evaluators' impressions after review of all available data, FWAS's ability to serve additional youth and operate more efficiently compared to past years resulted from partnering opportunities with the 21st CCLC program, additional Forth Worth Crime Control and Prevention District funding, increased school principal and teacher buy-in, and the quality of programming offered.

Summative/Outcome Findings

Academic	Over 90% of principals agreed that the program increased youth's awareness of the importance of learning, school attendance, and academic performance. Most surveyed parents agreed that their child did better in school (95%), and looked forward to going to school (96%) as a result of FWAS. Of surveyed youth, 75% agreed that they got better grades since they started attending FWAS. Of surveyed youth, 58% reported that they liked school more since being in FWAS, and 78% of parents of youth in grades 3 and above indicated that their child liked school more. Of surveyed youth, 69% said that they got homework help in FWAS and 59% said that they almost always or always finished homework at FWAS. Youth who got homework help at FWAS felt more strongly than those who did not get help that they liked school more since being in FWAS. Youth who participated in FWAS in year 5 only and in both years 4 and 5 increased their school attendance from year 4 to 5. For example, 60% of year 5 participants in grades 4 and 5 increased school attendance from below 95% in year 4 to 95% or greater in year 5, while 16% of middle school youth increased school attendance from below 95% in year 4 to 95% or greater in year 5. About two thirds of current fourth and fifth graders with TAKS scores of 1900 or less in 2004 improved their 2005 reading (69%) and math scores (63%). For those between 1901 and 2099 in 2004, 34% and 62% improved their 2005 reading and math scores, respectively. For current sixth to eighth graders with TAKS scores of 1900 or less in 2004, 57% improved their reading scores and 44% improved their math scores. For those between 1901 and 2099 in 2004, 50% and 24% improved their 2005 reading and math scores, respectively.
Prevention	Ninety-four percent of parents agreed that FWAS kept their child from getting in trouble and 45% also felt that their child would get into trouble if they were at home. Similarly, 69% of youth agreed that FWAS kept them from getting in trouble at home or in their neighborhood and 40% agreed that if they were at home they would be getting into trouble. Of parents who indicated that their child would have been getting into trouble at home, 74% felt that FWAS kept their child from getting into trouble. Of youth who indicated that they would be getting into trouble at home, 56% indicated they got into trouble less at FWAS. Youth who indicated that they would be getting into trouble if they were home were significantly more likely to say that there was nothing else to do in their neighborhood after school (p = .05).
Youth Development	The majority of parents (95%) felt that, as a result of FWAS, their child learned new activities they could do at home besides watching TV or playing video games. Parents reported more impact for younger youth (grades Pre-K–2) than older youth (grades 6–8). Youth indicated that they learned conflict resolution, sports/recreation, and art/drama skills. Of surveyed youth, 81% agreed FWAS helped them learn to work with other youth.

A Profile of the Evaluation of Fort Worth After School Program

Profile Information