You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.

www.HFRP.org

The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.

Terms of Use ▼

Program Description

Overview Fort Worth After School (FWAS) program is a community-driven initiative that provides after school programs for youth in Fort Worth, Texas. Program goals revolve around three areas: educational competence, physical and social development, and crime reduction. Services provided include homework assistance, tutorial, snacks, cultural and recreational activities, and mentoring.
Start Date September 2000
Scope local
Type after school
Location urban
Setting public school
Participants preschool through middle school students
Number of Sites/Grantees 52 schools (47 elementary schools and 5 middle schools, 2000–2004); 55 schools (48 elementary schools and 7 middle schools, 2004–2005)
Number Served In the first 5 years, 16,252 youth attended at least 1 year of FWAS. In year 5, funding was allocated to support 50–150 youth per day at each campus.
Components A Coordinating Board (CB) made up of city, school district, and community representatives oversees the program, while an administrative staff monitors day-to-day operations. Most programs operate 4 days per week (Monday–Thursday) from the time school is dismissed until 6 p.m. At many of the sites, the same youth participate all 4 days each week. At several sites, some youth participate 2 days per week, with a different group of youth participating the other 2 days. In a few cases, programs are offered on Fridays. At each site, youth receive a snack and an opportunity to complete homework. Youth receive a combination of enrichment activities and free play or recreation time. Service providers include programming in at least three of five areas: academic enrichment, sports and recreation, cultural and fine arts, community service, and character development. FWAS established a youth-to-staff ratio minimum for all sites: 15:1 for elementary schools and 20:1 for the middle school.

The CB's selection of school sites was based on a combination of factors including the percentage of youth eligible for the Federal Free Lunch Program and crime statistics for the area around the school. Each year the CB approves several agencies/organizations as potential service providers. Principals at participating schools either select one of these providers to offer the program on their campus or develop a “site-based” (self-managed) program. FWAS is staffed by a director and two program coordinators. A Parent Advisory Committee (PAC), formed in year 2, is made up of parents of program participants. In addition, in year 3, monthly networking meetings hosted by service providers, schools, and community organizations were established to give site coordinators, providers, and principals the opportunity to talk, share, and collaborate.
Funding Level The city of Fort Worth and Fort Worth Independent School District (FWISD) jointly provided a total of $2.2 million per year for the first 5 program years. Some sites used additional funding sources to serve more youth; some are supported in conjunction with FWISD 21st Century Community Centers (21st CCLC) programs.
Funding Sources City of Fort Worth Crime Control and Prevention District, FWISD, U.S. Department of Education 21st Century Community Learning Center funds


Evaluation

Overview The evaluation sought data about program inputs (the elements that make up FWAS, such as its leaders, facilities, and activities); outputs (the number of youth attending FWAS, the types of activities offered, and the number of program hours provided); and outcomes (what happens to youth, parents, and the community as a result of FWAS).
Evaluators Peter A. Witt, Terri King, Jin-Hyung Lee, Kristi Montadon, Lydia Justice, Joanne Oh, Billy Brown, Texas A&M University
Evaluations Profiled Fort Worth After School Program: “A Diamond in the Rough” First Year Evaluation

Fort Worth After School Second Year Evaluation

Fort Worth After School Third Year Evaluation

Fifth Year Evaluation: Fort Worth After School
Evaluations Planned Evaluations are ongoing.
Report Availability Witt, P. A., & King, T. (2001). Fort Worth After School program: “A diamond in the rough” first year evaluation. College Station, TX: Texas A&M University. Available at rptsweb.tamu.edu/Faculty/Witt/1Report%20Text.PDF

Witt, P. A., King, T., & Lee, J. H. (2002). Fort Worth After School second year evaluation. College Station: Texas A&M University. Available at rptsweb.tamu.edu/Faculty/Witt/
FortWorth2003/FWASEVALS.htm

Witt, P. A., King, T., & Montandoni, K. (2003). Fort Worth After School third year evaluation. College Station: Texas A&M University. Available at: rptsweb.tamu.edu/Faculty/Witt/
FortWorth2003/FWASEVALS.htm

Witt, P. A., King T., Justice, L., Oh, J., & Brown B. (2005). Fifth year evaluation: Fort Worth After School. College Station: Texas A&M University. Available at: rptsweb.tamu.edu/
Faculty/Witt/FortWorth2003/FWASEVALS.htm


Contacts

Evaluation Peter A. Witt
Elda K. Bradberry Recreation and Youth Development Chair
Department of Recreation, Park, and Tourism
Texas A&M University
2261 TAMU
College Station, TX 77843-2262
Tel: 979-845-7325
Email: pwitt@rpts.tamu.edu
Program Miguel Garcia
Director
Fort Worth After School
100 N. University NE 232
Fort Worth, TX 76107
Tel: 817-871-2369
Fax: 817-871-2893
Profile Updated May 24, 2006

Evaluation 1: “A Diamond in the Rough” First Year Evaluation


Evaluation Description

Evaluation Purpose To determine if FWAS was being successfully implemented and meeting its goals of youth educational competence, physical and social development, and crime reduction.
Evaluation Design Non-Experimental: Evaluators assessed stakeholders' program perceptions and satisfaction; FWAS attendance; and participants' social behavior, school liking, homework completion, and other academic abilities. A sampling plan to cover the range of service providers led to a designation of 15 target sites. Some data were collected for all 52 sites (attendance; site coordinator, staff, and principal surveys; and service provider and FWAS administrative staff focus groups), while other data were collected only at target sites (parent and youth surveys and interviews; principal and program coordinator focus groups; and site visits). Parent and youth interviews were done at three target sites. Surveys were completed by 332 parents (94 of youth in grades Pre-K–2 and 238 of youth in grades 3–8), 451 youth in grades 3 or above, 45 principals, 51 site coordinators, 156 FWAS staff, and each service provider administrator.
Data Collection Methods Document Review: FWAS staff provided data on program goals, activities, and costs.

Interviews/Focus Groups: Interviews and focus groups discussed program quality, outcomes, satisfaction, goals, and activities.

Observation: Each target site was visited at least 3 times to assess quality and outcomes.

Secondary Source/Data Review: Demographic and attendance data were collected during a 2-week audit in February.

Surveys/Questionnaires: Surveys asked about program goals and activities, quality, satisfaction, and outcomes.
Data Collection Timeframe Data were collected during the 2000–2001 school year.


Findings:
Formative/Process Findings

Activity Implementation A typical day consisted first of snack and math/reading tutorials and/or homework assistance. Youth then rotated through different activities, including book club, chess club, art club, music club, Spanish club, cooking, sports, fishing, and computers. Free play recreation opportunities were also provided at most sites.
Costs/Revenues Budgets ranged from $20,294 to $63,485 per site. Each child care slot was allocated $730 ($2.04 per hour per child at full capacity). At actual capacity, FWAS spent $2.60 per hour per child.

At a typical site, teachers and site coordinators were to be paid $20 per hour, while cafeteria workers, tutors, and assistants were to be paid $10 per hour. Different providers set their own pay rates around these standards.
Parent/Community Involvement Of parents surveyed, 70% agreed “a lot” that they received sufficient and appropriate information about the program from staff.
Program Context/Infrastructure Of parents surveyed, 84% agreed that if their child were not in FWAS, he or she would have had little to do (especially parents of elementary school youth). Only 52% of youth agreed with this statement, but in interviews, many said that there was nothing to do at other programs in their neighborhoods.

Of middle school parents, 35% reported that their child would have been without adult supervision if not in FWAS; smaller percentages of elementary school parents agreed with this statement (12% for Pre-K–2, 18% for grades 3–5).

Of youth surveyed, 41% reported feeling fear when walking in their neighborhood, while 94% agreed that they felt safe in FWAS. The majority of parents (97%) and principals (71%) agreed that FWAS provided a safe environment for participants,

Youth expressed some concern about the quality of the experience and their “confinement” at school as opposed to being able to be at home or out in their neighborhoods.

Most staff felt that supplies were adequate, but some noted this as in need of improvement.

Both youth and site coordinators generally considered the program's snacks a weakness.

The 52 schools had a total enrollment of 31,593 youth during the 2-week audit. Of youth in target schools, 81% were on the Federal Free/Reduced Lunch Program.
Program–School Linkages Some site coordinators disagreed that: principals kept them informed of important decisions (26%), a strong partnership existed between the school and FWAS (23%), and school staff followed through on commitments (17%). While principals and agency staff were more satisfied with partnerships, 22% of principals disagreed that FWAS reached out to school-day teachers.

One third of service providers disagreed that FWAS had adequate use of school space or facilities. Some sites had access to a wide array of school facilities, while others were more restricted. According to service providers, programs were often restricted to the cafeteria and outdoor play space, which was shared with other programs, and sometimes led to noisy and crowded conditions. In a few cases, programs had access to portable buildings and auditoriums, while library or computer facilities access was limited.
Recruitment/Participation According to FWAS attendance data, boys and girls were about equally represented; 52% of participants were Hispanic, 37% were African American, 8% were White, and 2% were other. In addition, 15% were in Pre-K and kindergarten, 26% were in grades 1–2, 52% were in grades 3–5, and 10% were in grades 6–8.

Sites differed in how youth were selected to participate. At some sites, youth were selected on a first-come, first-served basis, in others, youth were recommended by teachers based on their latchkey status or presumed need for social or academic development.
Satisfaction Of parents surveyed, 69% reported that their child was never or almost never bored during the program, while only 59% of youth gave the same answers.

Over 90% of surveyed parents said that they would recommend FWAS to others and would sign their child up again. Of youth surveyed, 47% said they would sign up again; most of the rest said they might, and only 15% said they would not recommend FWAS or sign up again. Older youth (grades 6–8) were less certain about signing up again than younger ones (grades 3–5). During interviews, some youth said that they did not tell others about FWAS for fear that more youth would join and they would get less attention and have less to do.
Staffing/Training FWAS had limited administrative and oversight team staff in its first 6 months.

Of parents surveyed, 84% thought that program leaders treated their child with respect, while 91% of youth agreed. In addition, 88% of parents thought that FWAS had good leaders.

Of principals surveyed, 96% agreed that FWAS staff properly supervised youth. The majority of principals and site coordinators also felt that staff set high expectations for youth and modeled responsible, positive behavior. However, they expressed concern in interviews regarding higher than desired staff turnover.

Coordinator and staff surveys indicated a need for more preservice and in-service training.


Summative/Outcome Findings

Academic All stakeholders indicated that homework completion was one of the best outcomes of FWAS participation. Three quarters of site coordinators indicated that FWAS provided an opportunity for youth to complete homework. In turn, parents and principals saw homework completion as impacting overall school-related performance.

Of youth surveyed, 49% reported always or almost always finishing homework at FWAS, and 63% reported that they always or almost always got help with their homework there.

Of principals surveyed, 80% reported increased homework completion at least sometimes, and about half agreed “very much” that FWAS improved the school's overall effectiveness (53%), students' attitude toward school (47%), and school attendance (49%).

Of parents surveyed, 61% agreed “a lot” that their children did better in school and looked forward more to going to school as a result of FWAS. Similarly, 59% of youth indicated that since they were in FWAS they liked school more; 69% of parents indicated the same.
Youth Development Sixty-two percent of youth surveyed agreed “a lot” that FWAS helped them learn to work with other youth; 71% of parents thought the same thing.

Of parents surveyed, 60% agreed “a lot” that their child learned new activities as a result of FWAS, including activities that they could also do at home instead of watching TV or playing video games.

Evaluation 2: Fort Worth After School Second Year Evaluation


Evaluation Description

Evaluation Purpose To determine if FWAS was being successfully implemented and meeting its goals of youth educational competence, physical and social development, and crime reduction.
Evaluation Design Non-Experimental: Evaluators assessed stakeholders' program perceptions and satisfaction; FWAS attendance; and participants' social behavior, school liking, homework completion, and other academic abilities. A sampling plan to cover the range of service providers led to a designation of 16 target sites. Some data were collected for all 52 sites (attendance; site coordinator, staff, and principal surveys; and service provider, FWAS administrative staff, and principal interviews), while other data were collected only for target sites (parent and youth surveys and interviews, program coordinator interviews, and site visits). Small group interviews were done with parents and youth at 3 target sites. Surveys were completed by 240 parents, 412 youth, 52 principals, 50 site coordinators, 199 program staff, full-service providers, and all CB members.
Data Collection Methods Document Review: FWAS staff provided data on program goals, activities, cost, and attendance.

Interviews/Focus Groups: Interviews and focus groups discussed program quality, outcomes, satisfaction, goals, and activities.

Observation: Each target site was visited at least 4 times to assess quality and outcomes.

Secondary Source/Data Review: Registration, demographic, and daily attendance data were collected from service providers.

Surveys/Questionnaires: Surveys asked about program goals and activities, quality, satisfaction, and outcomes.
Data Collection Timeframe Data were collected during the 2001–2002 school year.


Findings:
Formative/Process Findings

Costs/Revenues In year 2, funding per site ranged from $31,182 to $50,000. The average annual cost per child care slot was $840–$845. FWAS increased capacity use and lowered cost per child per hour at actual and full capacity from year 1 by 8% and 11% respectively. In year 2, full capacity cost per child per hour was $2.16, and actual capacity cost was $2.31 per child per hour.
Parent/Community Involvement Nearly all surveyed parents agreed that they received sufficient and appropriate information from program staff.

Various organizations implemented year 2 programs, including: Hispanic Youth Promoting Excellence, which brought high school students in to teach arts, improve motor skills, and inspire reading; Crime Prevention Resource Center, which held programs on violence, drugs, and character building; TCU Music Department, which brought in college music majors to teach Spanish songs; and TCU Latin Arts Festival, which provided workshops conducted by Latin American ballet and modern dance companies.
Program Context/Infrastructure In year 2, many programs switched from an outside service provider to maintaining and running the program internally to allow more control over program goals and processes.

In year 2, 84% of all target school youth were on the Federal Free/Reduced Lunch Program.

Surveyed parents were nearly unanimous in indicating that their child felt safe in FWAS; 86% of youth indicated the same. In addition, 29% of parents said that if not for FWAS, their child would have been without adult supervision at least some of the time.

Quality of snacks, while improved, continued to be an issue from the previous year.
Program–School Linkages Of site coordinators, 21% disagreed that principals informed them of important decisions.

Over 40% of programs hired school staff as site coordinators. At these sites, principals were more likely to rate FWAS staff as qualified and classroom management and discipline as performed very well, while site coordinators were more likely to say that school staff were supportive of FWAS and that teachers were more willing to collaborate and provide information on homework. Further, programs that hired school staff were more likely to have a school-wide mechanism or strategy for communication between school-day teachers and FWAS staff about homework assignments (88% of teachers employed by FWAS agreed vs. 63% of teachers not working at FWAS).

Sixty percent of schools directly employed the site coordinator. In these cases, all site coordinators agreed that there was a strong partnership between FWAS and the school vs. 71% of site coordinators not employed by schools. Similarly, site coordinators reported differences in the extent to which schools informed FWAS staff of important decisions depending on whether the site coordinator was employed by the school.

Programs that shared school cafeteria or gym space with other programs sometimes experienced noisy and crowded conditions. FWAS staff worked with principals and providers to ensure heightened access to classrooms, libraries, and computer facilities. Programs that hired school staff helped FWAS gain access to school facilities.
Recruitment/Participation In year 2, 2,600 program slots were available (50 per school). Daily capacity use was 94%, with 4,728 youth registered in FWAS (182% of slots, indicating an 82% enrollment turnover). The average child enrolled for 87 days, or 67% of possible days.

Of FWAS participants, 35% were African American, 53% were Hispanic, and 9% were White; 51% were female and 49% were male. Nearly 11% were in Pre-K or kindergarten, 22% were in grades 1–2, 56% were in grades 3–5, and 12% were in grades 6–8.
Satisfaction Almost all surveyed parents (98%) said that they would sign their child up for FWAS again or recommend it to others. Fewer youth said that they would recommend it (39%) or sign up again (38%). However, the majority of youth surveyed (91%) liked FWAS. A common reason given in interviews for why youth did not want to participate again was that they did not want to be at school during the school day plus after school.
Staffing/Training Most site coordinators and staff agreed that training was adequate, but in focus groups, agency administrators and site coordinators indicated a need for more training in creative discipline/classroom management, First Aid/CPR, and programming ideas.

All site coordinators agreed at least somewhat that staff set high expectations for youth. Most principals felt that providers did well in hiring staff, but some had concerns about turnover.

Almost all surveyed parents thought that FWAS had good leaders and that leaders treated their child with respect. Similarly, the majority of youth thought that FWAS had good leaders (92%) and that FWAS adults always or almost always cared about them (70%).


Summative/Outcome Findings

Academic Over half of surveyed youth said that they always or almost always had time to finish homework in FWAS (52%) and got help with homework (55%); 67% of parents agreed “a lot” that their child did better on homework due to FWAS.

About half of principals strongly agreed that FWAS increased youth awareness of the importance of learning (51%), school attendance (51%), and academic performance (43%).

Of those surveyed, 43% of youth and 53% of parents said that youth liked school more and 63% of parents agreed “a lot” that their child did better in school due to FWAS.
Youth Development The majority of youth (80%) and parents (71%) surveyed agreed that FWAS helped youth learn to work with other youth.

Evaluation 3: Fort Worth After School Third Year Evaluation


Evaluation Description

Evaluation Purpose To determine if FWAS was being successfully implemented and meeting its goals of youth educational competence, physical and social development, and crime reduction.
Evaluation Design Non-Experimental: Evaluators assessed stakeholders' program perceptions and satisfaction; FWAS attendance; and participants' social behavior, school liking, homework completion, and other academic abilities. A sampling plan to cover the range of service providers led to a designation of 16 target sites. Some data were collected for all 52 sites (attendance; site coordinator, staff and principal surveys), while other data were collected only from target sites (parent and youth surveys and interviews; site visits). Surveys were completed in February by 358 parents, 492 youth in grades 3 and above, 52 principals, 52 site coordinators, 183 program staff, and all CB members. All principals and site staff who completed the February survey completed an additional survey in May. Interviews were ongoing with all of these stakeholders (except CB members) throughout the year.
Data Collection Methods Document Review: FWAS staff provided data on program goals, activities, cost, and attendance.

Interviews/Focus Groups: Interviews discussed program quality, outcomes, satisfaction, goals, and activities.

Observation: Each target site was visited at least 4 times to assess quality and outcomes.

Secondary Source/Data Review: Registration, demographic, and daily attendance data were collected from service providers.

Surveys/Questionnaires: Surveys asked about program goals and activities, quality, satisfaction, and outcomes.
Data Collection Timeframe Data were collected during the 2002–2003 school year.


Findings:
Formative/Process Findings

Activity Implementation Site coordinators reported that in year 3, 100% of sites provided homework assistance, fine arts, and character development/life skills; 98% provided academic enrichment and recreation/sports; and 88% provided community service.

During site visits, the following activities were observed: homework assistance (100% of sites), sports/recreation (100%), fine arts (94%), academic enrichment (75%), character development (63%), and community service (6%).

PAC members were impressed with the variety of activities, although items of concern expressed to the CB included a desire to have FWAS meet Monday through Friday (not just 4 days per week) and to make programs more fun.

The percentages of principals rating elements of FWAS as “very important” to include as part of the program at their site were: homework assistance/tutoring (78%); academic enrichment and culture/fine arts (59% each); character development (52%); sports/recreation (29%); and community service (27%). At sites where principals rated sports/recreation as very important, 50% indicated that FWAS performed very well in this area. Corresponding performance scores for the other areas were: cultural/fine arts (69%), homework assistance/tutoring (61%), academic enrichment (59%), character development (56%), and community service (54%). While importance ratings changed little from year 2, performance ratings improved for all areas except sports/recreation. Over the first 3 years, principals made changes in providers to meet their perceptions of important elements and then to maximize performance in each area.

About two thirds of youth surveyed (66%) indicated that they always or almost always got homework help at FWAS, while 100% of coordinators reported that FWAS provided opportunities to complete homework always or almost always and 76% of principals agreed that the provider performed well at providing homework assistance/tutoring. In addition, PAC members were grateful that FWAS provided opportunities to complete homework, although a desire for more tutoring was expressed.
Costs/Revenues Site budgets ranged from $39,153 to $47,790 ($42,297 on average), totaling $2,199,446. Several sites had funds from other sources that enabled them to serve additional youth (two middle school sites only received $18,000 in FWAS funds because 21st CCLC funds were also available at these sites). The cost per hour per child was $2.22 at full capacity and $2.17 at actual capacity.

PAC members indicated that they would be willing to pay for FWAS.
Parent/Community Involvement The majority of parents surveyed (96%) agreed that program staff supplied them with good information about FWAS.

In interviews, youth indicated that their parents were not always able to help them with homework due to language barriers, work schedule, or abilities.

In year 3, several organizations implemented programs. 4-H Club activities (e.g., gardening, self-esteem, student government, character building, water management) were provided at almost half of sites. Youth at two sites participated in the National Block Kids Building Contest, during which youth learned about construction profession. The Tarrant Area Food Bank's Kids Café provided hot meals to youth and staff at three sites. Sisters with Pride, a self-esteem and community service club, served girls at one site. Volunteers at five sites assisted with reading and homework through TCU America Reads.
Program Context/Infrastructure Parent survey responses indicated that if not in FWAS, 69% of youth would have had adult supervision almost all of the time, 19% for some of the time, 9% would have been alone, and 3% would have been with other youth but no adults. Parents indicated that, to be supervised, youth would have had to go to work with them or that supervision might have been provided by a sleeping adult. PAC members were grateful that their children were cared for and not alone after school.

The majority of parents surveyed (91%) agreed that if their child was not in FWAS, he or she would have had little to do. Just over half of youth surveyed (53%) agreed that there was nothing to do after school. In focus groups, when asked what they would have been doing after school if not in FWAS, youth said that they would have been home bored; playing outside; riding bikes, scooters, or skate boards; playing video games; watching TV; or just “hanging out.” When youth were asked about other programs in the neighborhood, they said that there was little else for them to do.

Of youth surveyed, 41% indicated some fear when walking in their neighborhood; those in grades 3–5 reported more fear (42%) than those in grades 6–8 (32%).

Of those surveyed, 90% of youth felt safe, and 99% of parents said that their child felt safe in FWAS. Similarly, 98% of principals and 100% of site coordinators said that FWAS provided a safe environment for youth. Youth who felt most afraid in their neighborhood felt safest in FWAS. When asked in interviews how FWAS made them feel safe, youth said that adults were always around and gave them attention, and that they felt safe being in the school building. Youth at several sites said that staff kept a list of adults who were allowed to sign them out, which gave them a sense of being protected.

The majority of parents (99%) and youth (94%) agreed that there were rules that youth must follow in FWAS. In interviews, youth said that rules helped them feel safe.

Only 48% of youth surveyed always or almost always liked the snack, 37% sometimes liked it, and 14% never or almost never liked it. In contrast, the majority of site coordinators believed that youth liked the snack (88%) and that there was a variety of snacks provided (89%).

Most staff surveyed agreed that they were provided with adequate space/facilities (96%) and supplies/materials (92%).

Eighty-six percent of students in target schools were on the Federal Free/Reduced Lunch Program. Schools had a total enrollment of 30,816.

Many schools changed providers in years 2 and 3, often because principals sought providers to match their desired goals. In many cases, principals decided to become site-based (self-managed) to improve program development, management, and congruence with program goals. As FWAS went into year 4, the number of providers diminished from 11 to 6 (site-based providers are counted as a single category). Three agencies that had 16 sites between them in year 1 either had partial service status or were no longer involved with FWAS by year 3. These changes were due to principals' perceptions that the agency could not offer the full range of services desired, used too much funding for overhead, and/or did not provide the appropriate level of administrative oversight.
Program–School Linkages Year 3 networking meetings had an average attendance of 20–30 people, including site coordinators, service provider administrators, and principals. Topics and agendas included standards and guidelines, funding and grants, evaluation highlights, best practices, service provider overviews, resources available to programs, and collaboration opportunities.

Forty-two percent of programs hired school staff as site coordinators, and 58% of FWAS staff indicated that they were employed as school staff. All site-based programs employed teachers as site coordinators and staff. At 54% of schools, the site coordinator was also on the school staff.

The majority of agency staff agreed that they had regular contact with principals (96%) and that FWAS was part of an ongoing site-based committee or a separate committee established to deal with after school program issues (93%).

The majority of site coordinators (88%–96%) agreed that the principal was supportive of FWAS; they had regular contact with teachers to identify youth's needs and incorporate them into FWAS activities; a strong partnership existed between the school and FWAS; teachers were willing to collaborate with FWAS staff and provide information about homework; FWAS staff were supportive of FWAS; and the school informed them of important decisions. Coordinators employed by the school tended to agree with these items more often than those not employed by the school.

Of principals surveyed, 86% indicated that a mechanism was in place for communication between teachers and FWAS staff about homework assignments, and 81% indicated that the site-based committee included FWAS staff and/or FWAS program agenda items in regularly scheduled management meetings.

Of principals surveyed, 51% reported visiting FWAS at least once a week, 31% one or fewer times a month, 14% once a month, and 4% 1–2 times a semester; site based programs were visited more frequently. Of site coordinators, 35% reported that principals and teachers visited at least once a week, 35% a few times a month, 6% about once a month, 12% 1–2 times a semester, 2% less than once a semester, and 10% never.

In sites where both the site coordinator and other program staff were also on the school staff, 70% of principals reported that FWAS performed very well in communication with school staff. The corresponding percentages for other types of sites were: 57% in sites where only the site coordinator was on the school staff, 22% in sites where employee school staff but not the site coordinator was on the school staff, and 25% in sites where FWAS employed no school staff.

In years 1–3, sites had good cafeteria, gym, and auditorium access. In year 3, 96% to 98% had access to these school facilities: 64% had frequent auditorium access (every day or 2-3 times per week), 88% had frequent gym access, and 94% had frequent cafeteria access. Sites also had increased access to other facilities: In year 3, 81% had classroom access (71% frequently), 80% had library access (49% frequently), and 69% had computer lab access (39% frequently). Increased access, which led to more flexibility and programming variety, was due partly to more site-based programs, where school staff had more confidence in facility use. In some cases, principals gained confidence in providers and/or FWAS staff worked with providers and principals to increase access. Some programs failed to work closely with schools to get permission to use facilities. In a few cases, lack of trained staff prohibited facility use. Several programs shared cafeteria or gym space with other programs, which sometimes led to noisy and crowded conditions.
Recruitment/Participation The number of funded slots at each site was set at 50, for a total of 2,600 slots at the 52 sites. Slots were only available for 8% of the total students across schools.

In year 3, 47% of participants were male and 53% were female. In terms of ethnicity, 54% were Hispanic, 35% were African American, 8% were White, 1% were Asian, less than 1% were Native American, and 1% reported their ethnicity to be other.

FWAS attracted an increasing percentage of older youth from years 1–3, with youth in Pre-K to grade 2 decreasing from 41% to 28% of total participants, grades 3–5 increasing from 52% to 59%, and grades 6–8 increasing from 8% to 13%. Each grade from Pre-K to grade 5 had a fairly steady decline; the higher the grade, the lower the attendance (e.g., first graders had lower attendance than kindergarteners, etc.). Grades 6–8 varied by year, declining by grade in year 3, while year 2 declined for grade 6 but increased for grades 7 and 8.

Efforts were made in year 3 to standardize how youth were selected into FWAS; providers were urged to first take previous-year enrollees, then latchkey youth and those in need of social skill or academic development.

Programs operated from 109 to 162 days per site in year 3; 73% operated 109–125 days, and 27% 126–162 days, with an average of 126 and a total of 6,588 program days.

A total of 4,651 youth attended FWAS at least 1 day in year 3 (60–257 youth per site); this represented 175% of slots, indicating some enrollment turnover. The actual number of attendance days was 334,898 in year 3, a 6% increase from year 2. The possible number of attendance days decreased 3% from year 2 to 384,282 in year 3.

The percentage of slots utilized on a daily basis rose from 94% in year 2 to 102% in year 3. In year 3, more programs utilized more than their 50 allocated slots per day than in the previous year (49% vs. 33%), with fewer programs utilizing 39 or fewer slots per day (10% vs. 23%).

At most sites, the same youth participated all 4 days each week.
Satisfaction Over 95% of parents said that they would recommend FWAS to others or sign up their child again. For youth, 48% said that they would recommend FWAS to others, and 47% said that they would sign up again (35% and 40% said “maybe,” respectively). In interviews, some youth said that they did not tell others about FWAS because they did not want more youth to join, feeling that they would get less attention or have less to do. Others said that they would rather be at home, because they did not like to be at school through the school day plus after school.

Of parents surveyed, 92% indicated that they really liked FWAS, 91% agreed “a lot” that they liked being able to send their child to FWAS, and 87% agreed “a lot” that their child liked it. Similarly, PAC members were pleased with FWAS and grateful that their youth were given opportunities to grow. In contrast, 65% of youth surveyed indicated that they really liked FWAS.

Of parents surveyed, 84% reported that their child never or almost never felt bored during the program, compared to 60% of youth.

Of youth surveyed, 68% indicated that they felt good about themselves always or almost always when at FWAS.
Staffing/Training Major trainings held for FWAS staff in year 3 included: (a) FWAS Programming, which included such programming issues as attendance procedures and the importance of enrolling latchkey youth; (b) FWAS Attendance & Budget (2 trainings, one for site-based campuses and a second for service providers), which discussed procedures for data input, submission deadlines, and budget updates; and (c) After School Quality, which discussed creative programming that could be implemented in programs.

Of those surveyed, 84% of site coordinators and 92% of staff reported that training was at least somewhat adequate.

Of site coordinators surveyed, 39% reported receiving 15+ hours of training, 22% reported 11–14 hours, 16% reported 6–10 hours, 12% reported 3–5 hours, and 6% each reported 1–2 hours or none. For staff, 32% reported 15+ hours, 15% reported 11–14 hours, 14% reported 6–10 hours, 17% reported 3–5 hours, 11% reported 1–2 hours, and 12% reported none.

Of parents surveyed, 96% agreed that there were adults in FWAS whom their child respected, and 94% of youth agreed with this. Similarly, 82% of youth felt that FWAS adults always or almost always cared about them.

Almost 100% of parents agreed that FWAS had good leaders. PAC members were also impressed with staff quality.

All site coordinators felt that staff set high expectations for youth.

Of principals surveyed, 83% felt that the provider performed well in hiring qualified staff; they were more positive about staff quality when programs were site-based.

In most cases, service providers were able to retain staff, but in some cases there was greater than desired turnover. Principals and site coordinators indicated that providers should make continuing efforts to hire staff that would be available for the entire school year and have experience working with youth.


Summative/Outcome Findings

Academic The majority of parents (98%) reported that as a result of FWAS, their child was doing better at homework, although only 54% of youth indicated that they always or almost always finished homework at FWAS.

The majority of principals surveyed agreed that FWAS performed well in improving youth's: awareness of the importance of learning (82%), school attendance (77%), and academic performance (81%).

Of parents surveyed, 90% agreed that their child did better at school, and 98% agreed that their child looked forward to going to school due to FWAS. Of youth surveyed, 55% indicated that since they were in FWAS they liked school more; 77% of parents indicated the same.
Youth Development Eighty-two percent of youth agreed that FWAS helped them learn to work with other youth; 98% of parents reported the same. During interviews, parents confirmed that they thought youth were learning to work better with others.

The majority of parents (94%) agreed that their child learned new activities in FWAS that they could do at home instead of watching TV or playing video games.

Evaluation 4: Fifth Year Evaluation: Fort Worth After School


Evaluation Description

Evaluation Purpose To determine which elements are necessary to offer a quality program, and what benefits for youth are achieved through participating in the program.
Evaluation Design Non-Experimental: Evaluators assessed stakeholders' program perceptions and satisfaction; FWAS attendance; and participants' social behavior, school liking, homework completion, and other academic abilities. Surveys were collected from 991 parents at 47 sites, 1,693 youth at 48 sites, 51 principals, 40 site coordinators, and 191 program staff at 45 sites. Program attendance, academic, school attendance, and other data were collected on all participants over the first 5 years to explore the impact of FWAS attendance on academic performance and school attendance. Site visits and program staff interviews were conducted at all sites.
Data Collection Methods Document Review: FWAS staff provided data on program goals, activities, cost, and attendance.

Interviews/Focus Groups: Interviews and focus groups discussed program quality, outcomes, satisfaction, goals, and activities.

Observation: Site visits were made to all sites at least 3 times during the school year to assess program quality.

Secondary Source/Data Review: Participants' school attendance and test-score data were collected from schools. In addition, program registration, participant demographic, and daily attendance data were collected from service providers.

Surveys/Questionnaires: Surveys asked about program goals, activities, quality, satisfaction, and outcomes.

Tests/Assessments: Scores on the Texas Assessment of Knowledge and Skills (TAKS™), the statewide assessment program, were collected for FWAS participants.
Data Collection Timeframe Data were collected during the 2004–2005 program year.


Findings:
Formative/Process Findings

Costs/Revenues The cost per hour declined from $2.62 per child per hour in year 1 to $1.67 in year 5.

In year 5, FWAS benefited from in-kind donations valued at $102,860 from 37 sources and 1,500 volunteer hours from 53 organizations.
Program Context/Infrastructure When surveyed, 38% of youth agreed that they were afraid when walking in their neighborhoods. Youth in grades 3–5 reported more fear (38%) than those in grades 6–8 (30%). In contrast, 92% of youth agreed that they felt safe in FWAS.

Parents indicated on surveys that if their children were not in FWAS, 62% would have had adult supervision almost all of the time, with another 18% having adult supervision some of the time. In some cases, parents indicated that to be supervised, youth would have had to go to work with them or that supervision might have been provided by a sleeping adult. Without FWAS, 20% of parents felt that their child would have lacked adult supervision after school.

Of youth surveyed, 49% agreed that there was nothing to do during after school hours besides FWAS, and 75% of parents agreed that if not in FWAS, their child would have had little to do.
Program–School Linkages From years 1–5, the number of schools managing their own programs increased from 3 to 22 (the number of external providers decreased from 10 to 5). This change was due to several factors. For example, principals felt that if programs were self-managed, they could better align FWAS with school goals; teachers would be well-positioned to know the youth and their needs; and they could allow the use of more school facilities. However, the number of site-based programs leveled off by year 5, as external providers made adjustments to improve quality and work with schools. As a result, principals indicated greater satisfaction with their providers in year 5, and few additional changes were made.

Teacher involvement in FWAS usually led to greater confidence by principals and school staff that school facilities would not be abused. Consequently, some external providers hired teachers as site supervisors or program staff to ensure stronger ties to the school and greater confidence in program staff's ability to care for school facilities.
Recruitment/Participation Of 5th year registrants, 51% were girls and 49% were boys. Twenty-seven percent were in grades Pre-K–2, 52% were in grades 3–5, and 22% were in grades 6–8. Fifty-nine percent were Hispanic, 34% were African American, 6% were White, and 2% were other ethnicities. In addition, 30% were classified as Limited English Proficient; 92% were eligible for the Federal Free Lunch Program; and 56% were considered at-risk of school failure, based on Forth Worth Independent School District criteria.

In year 5, there were 3,695 FWAS attendance slots with 6,364 youth enrolled at sometime during the year, with a total of over 1.4 million child hours of attendance.

FWAS operated an average of 132 days across sites during year 5.

Since the program's inception, 16,252 different youth attended at least 1 year of FWAS, with 67% attending only 1 year, 23% enrolled for 2; 8% enrolled for 3, and 3% enrolled for 4-5. Approximately 50% of year 5 participants attended 2 or more years.

FWAS operated at over 100% of projected daily capacity in years 3–5 (105% in year 5). Youth participated in the majority of days enrolled (92% in year 5). As youth moved from elementary to middle school, the average number of days attended in a given year decreased.
Satisfaction Of youth surveyed, 72% agreed that they liked FWAS, 73% indicated that they felt good about themselves when at FWAS, and only 14% indicated that they felt bored during FWAS.

Over 95% of surveyed parents reported that they would recommend FWAS to others, and 93% would sign their child up again. Some parents who said they wouldn't sign their child up noted that their child would be moving on to middle or high school. Youth in grades 3–4 were more likely to say that they wanted to sign up next year than those in grades 6–7 (63% vs. 44%), with youth in grades 5 and 8 even less committed (39% and 30%).

The most common responses to what youth liked best about FWAS were fun games and activities, improving social skills, sports, playing outside (grades 3–5), teachers/staff (grades 3–5), technology classes (grades 6–8), and variety of activities (grades 6–8). The most common responses to what youth would change were snacks, nothing, more fun activities and classes, and less discipline.
Staffing/Training According to evaluators' impressions after review of all available data, FWAS's ability to serve additional youth and operate more efficiently compared to past years resulted from partnering opportunities with the 21st CCLC program, additional Forth Worth Crime Control and Prevention District funding, increased school principal and teacher buy-in, and the quality of programming offered.


Summative/Outcome Findings

Academic Over 90% of principals agreed that the program increased youth's awareness of the importance of learning, school attendance, and academic performance.

Most surveyed parents agreed that their child did better in school (95%), and looked forward to going to school (96%) as a result of FWAS.

Of surveyed youth, 75% agreed that they got better grades since they started attending FWAS.

Of surveyed youth, 58% reported that they liked school more since being in FWAS, and 78% of parents of youth in grades 3 and above indicated that their child liked school more.

Of surveyed youth, 69% said that they got homework help in FWAS and 59% said that they almost always or always finished homework at FWAS. Youth who got homework help at FWAS felt more strongly than those who did not get help that they liked school more since being in FWAS.

Youth who participated in FWAS in year 5 only and in both years 4 and 5 increased their school attendance from year 4 to 5. For example, 60% of year 5 participants in grades 4 and 5 increased school attendance from below 95% in year 4 to 95% or greater in year 5, while 16% of middle school youth increased school attendance from below 95% in year 4 to 95% or greater in year 5.

About two thirds of current fourth and fifth graders with TAKS scores of 1900 or less in 2004 improved their 2005 reading (69%) and math scores (63%). For those between 1901 and 2099 in 2004, 34% and 62% improved their 2005 reading and math scores, respectively. For current sixth to eighth graders with TAKS scores of 1900 or less in 2004, 57% improved their reading scores and 44% improved their math scores. For those between 1901 and 2099 in 2004, 50% and 24% improved their 2005 reading and math scores, respectively.
Prevention Ninety-four percent of parents agreed that FWAS kept their child from getting in trouble and 45% also felt that their child would get into trouble if they were at home. Similarly, 69% of youth agreed that FWAS kept them from getting in trouble at home or in their neighborhood and 40% agreed that if they were at home they would be getting into trouble.

Of parents who indicated that their child would have been getting into trouble at home, 74% felt that FWAS kept their child from getting into trouble. Of youth who indicated that they would be getting into trouble at home, 56% indicated they got into trouble less at FWAS. Youth who indicated that they would be getting into trouble if they were home were significantly more likely to say that there was nothing else to do in their neighborhood after school (p = .05).
Youth Development The majority of parents (95%) felt that, as a result of FWAS, their child learned new activities they could do at home besides watching TV or playing video games. Parents reported more impact for younger youth (grades Pre-K–2) than older youth (grades 6–8).

Youth indicated that they learned conflict resolution, sports/recreation, and art/drama skills.

Of surveyed youth, 81% agreed FWAS helped them learn to work with other youth.

© 2016 Presidents and Fellows of Harvard College
Published by Harvard Family Research Project