Search Publications | The Evaluation Exchange | FINE Network | OST Database

The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.

A Profile of the Evaluation of Virtual Y

Program Description

Overview	Virtual Y is an initiative that brings YMCA after school programs and staff into New York City public elementary schools to provide youth with safe, fun-filled, and challenging activities between the hours of 3 and 6pm. It offers support for classroom learning by extending the school day and helping children achieve reading proficiency through literacy-based activities. Virtual Y is committed to building the spirit, mind, and body of all participating children and to enriching community, family, and school.
Start Date	1997
Scope	local
Type	after school
Location	urban
Setting	public school
Participants	elementary students

Number of Sites/Grantees	66 Virtual Ys during 1997–1998, 100 Virtual Ys during 1998–1999, and 104 Virtual Ys during 1999–2000
Number Served	8,000 during the 1999–2000 school year
Components	Virtual Y offers support for classroom learning by extending the school day and helping children achieve reading proficiency through literacy-based activities. Each site enrolls at least 50 second through fourth graders at no cost to students. Children attend five days a week. All activities include a reading enhancement component. Children are also taught the four core YMCA values: respect, responsibility, honesty, and caring.
Funding Level	$5.5 million
Funding Sources	Private donors, both individual and corporate including: AT&T, Chase Manhattan Bank, Kraft, NYNEX, the Equitable Foundation, and the Calder Foundation

Evaluation

Overview	Annual evaluation reports have been produced each year since the program's inception. Four reports are issued each year: needs assessment, implementation, behavioral outcomes, and academic outcomes.
Evaluator	National Center for Schools and Communities, Fordham University
Evaluations Profiled	1999–00 Program Implementation Report Impact of the Virtual Y on Children's Classroom Behavior: Program Year 1999–00 Preliminary Analysis of Virtual Y After-School Program Participants' Patterns of School Attendance and Academic Performance, Final Evaluation Report, Program Year 1999–2000
Evaluations Planned	Fordham University is conducting additional analysis of the Virtual Y evaluation data using Hierarchical Linear Modeling. This new report should be available in the fall of 2003.
Report Availability	Foley, E. M., & Eddins, G. (2001). Preliminary analysis of Virtual Y After-School Program participants' patterns of school attendance and academic performance. Final evaluation report program year 1999–2000. New York: National Center for Schools and Communities, Fordham University. Foley, E. M., & Eddins, G. (2001). Impact of the Virtual Y on children's classroom behavior: Program year 1999–2000. New York: National Center for Schools and Communities, Fordham University. Foley, E. M., & Eddins, G. (2000). Virtual Y: 1999–00 program implementation report. New York: National Center for Schools and Communities, Fordham University.

Contacts

Evaluation	Gillian Eddins Senior Research Associate National Center for Schools and Communities Fordham University 33 West 60th St, 8th floor New York, NY 10023 Tel: 212-636-6697 Email: eddins@fordham.edu
Program	Wanda Milton YMCA of Greater New York 333 Seventh Avenue, 15th Floor New York, NY 10001 Tel: 212-630-9697 Fax: 212-630-9725 Email: wmilton@ymcanyc.org
Profile Updated	July 9, 2003

Evaluation 2: Impact of the Virtual Y on Children's Classroom Behavior: Program Year 1999–2000

Evaluation Description

Evaluation Purpose	To examine the impact of the Virtual Y on children's classroom behavior.
Evaluation Design	Quasi-Experimental: Data on between 500 and 650 children's classroom behaviors were collected and compared during each year of the program. The research sample in each year was the group of children for whom there were parent permission for the evaluation process and both a pre- and post-program teacher rating. The group of students with valid pre- and post-program data was approximately 20% to 30% of children with parental permission for each of the three years. Given fluctuations in available data, changes in program sites, and normal student turnover, very few children (fewer than 50) were included among study participants in multiple years. The small number ruled out the possibility of meaningful longitudinal analysis. In year 3, it was possible to assess the study sample against the full population of Virtual Y participants and nonparticipants. Compared to the larger population of the school, study children were poorer and more likely to be minorities and of limited English proficiency (LEP). They were also more likely to be female and recent immigrants. They were younger for their grade, and they had better attendance records than non-program participants. The evaluators caution that because of these differences they could not rule out selection effects that may have accounted for pre- and post-program differences detected in their sample. To reduce the likelihood that these selection effects might bias any discovered impacts, the evaluators controlled for LEP status, gender, race, age, and prior attendance in their analyses. The regression analyses also controlled for grade-level age effects to assess the extent to which students' maturation may have been driving any discovered program effects, and maturation effects were found unlikely to have been a major factor.
Data Collection Methods	Tests/Assessments: The Teacher-Child Rating Scale (Hightower, A. D., et al.) was used to obtain pretest and posttest ratings of Virtual Y participants from their classroom teachers. The instrument is composed of one scale measuring overall classroom behavior, and seven subscales measuring the following behaviors: acting out, shyness and anxiety, learning skills, frustration tolerance, assertive social skills, task orientation, and peer social skills. References Hightower, A. D., Work, W. C., Cowen, E. L., Lotyczewski, B. S., Spinnell, A. P., Guare, J. C., et al. (1986). The Teacher-Child Rating Scale: A brief objective measure of elementary children's school problem behaviors and competencies. School Psychology Review, 15, 393–409.
Data Collection Timeframe	Data were collected at the beginning and end of each of the school years: 1997–1998 (year 1), 1998–1999 (year 2), and 1999–2000 (year 3).

Findings:
Formative/Process Findings

Activity Implementation	Quality activities and enhanced program resources, such as extra books were found to be associated with better outcomes in Virtual Y programs.
Parent/Community Involvement	Careful communication with both the home and host school (as rated by parents) was found to be associated with better outcomes in Virtual Y programs.
Recruitment/Participation	Teachers reported a high proportion of Virtual Y participants as having behavioral problems. According to their school-day classroom teachers, 85% of participants in year 1, 91% of participants in year 2, and 89% of participants in year 3 evidenced some behavioral problem at the time of pre-program testing. The problems were most frequently reported to be in the mild range. Across years, the highest preprogram scale scores of participants (indicating the greatest problems) were consistently in the areas of task motivation, frustration, tolerance, and learning skills. The most consistently troubling individual behaviors for children in year 1 and year 2 were organizing themselves and functioning with distractions; for year 3, the most troubling individual behaviors were functioning well with distractions and ignoring teasing. The least troubling areas for children in year 1 and year 3 were nervous behavior and sadness. In year 2, the least troubling areas for children were nervous behavior and overly aggressive behavior towards peers (fighting).
Staffing/Training	Regression analysis identified the following features of the Virtual Y associated with better outcomes: positive student and staff relationships; staff perception that their workload is manageable; higher percentages of staff who are non-Board of Education employees; staff who rely especially on YMCA trainings in developing their lessons; enhanced program resources such as extra staff; higher percentages of staff with four-year college degrees; lower student to staff ratios; staff who speak more than one language; higher percentages of staff who make use of fellow counselors' suggestions; and higher percentages of staff who are teachers.

Summative/Outcome Findings

Academic	In all years, the data showed statistically significant (p<. 01) and moderate to large improvements on the learning skills subscale.
Youth Development	In all years, the data showed statistically significant (p<. 01) and moderate to large improvements on all seven subscales (acting out, shyness and anxiety, frustration tolerance, assertive social skills, task orientation, and peer social skills) and on the overall behavior scale. The effects were found for children at each level of behavioral difficulty, from mild to severe.

A Profile of the Evaluation of Virtual Y

Profile Information