You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.

www.HFRP.org

The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.

Terms of Use ▼

Program Description

Overview Virtual Y is an initiative that brings YMCA after school programs and staff into New York City public elementary schools to provide youth with safe, fun-filled, and challenging activities between the hours of 3 and 6pm. It offers support for classroom learning by extending the school day and helping children achieve reading proficiency through literacy-based activities. Virtual Y is committed to building the spirit, mind, and body of all participating children and to enriching community, family, and school.
Start Date 1997
Scope local
Type after school
Location urban
Setting public school
Participants elementary students
Number of Sites/Grantees 66 Virtual Ys during 1997–1998, 100 Virtual Ys during 1998–1999, and 104 Virtual Ys during 1999–2000
Number Served 8,000 during the 1999–2000 school year
Components Virtual Y offers support for classroom learning by extending the school day and helping children achieve reading proficiency through literacy-based activities. Each site enrolls at least 50 second through fourth graders at no cost to students. Children attend five days a week. All activities include a reading enhancement component. Children are also taught the four core YMCA values: respect, responsibility, honesty, and caring.
Funding Level $5.5 million
Funding Sources Private donors, both individual and corporate including: AT&T, Chase Manhattan Bank, Kraft, NYNEX, the Equitable Foundation, and the Calder Foundation


Evaluation

Overview Annual evaluation reports have been produced each year since the program's inception. Four reports are issued each year: needs assessment, implementation, behavioral outcomes, and academic outcomes.
Evaluator National Center for Schools and Communities, Fordham University
Evaluations Profiled 1999–00 Program Implementation Report

Impact of the Virtual Y on Children's Classroom Behavior: Program Year 1999–00

Preliminary Analysis of Virtual Y After-School Program Participants' Patterns of School Attendance and Academic Performance, Final Evaluation Report, Program Year 1999–2000
Evaluations Planned Fordham University is conducting additional analysis of the Virtual Y evaluation data using Hierarchical Linear Modeling. This new report should be available in the fall of 2003.
Report Availability Foley, E. M., & Eddins, G. (2001). Preliminary analysis of Virtual Y After-School Program participants' patterns of school attendance and academic performance. Final evaluation report program year 1999–2000. New York: National Center for Schools and Communities, Fordham University.

Foley, E. M., & Eddins, G. (2001). Impact of the Virtual Y on children's classroom behavior: Program year 1999–2000. New York: National Center for Schools and Communities, Fordham University.

Foley, E. M., & Eddins, G. (2000). Virtual Y: 1999–00 program implementation report. New York: National Center for Schools and Communities, Fordham University.


Contacts

Evaluation Gillian Eddins
Senior Research Associate
National Center for Schools and Communities
Fordham University
33 West 60th St, 8th floor
New York, NY 10023
Tel: 212-636-6697
Email: eddins@fordham.edu
Program Wanda Milton
YMCA of Greater New York
333 Seventh Avenue, 15th Floor
New York, NY 10001
Tel: 212-630-9697
Fax: 212-630-9725
Email: wmilton@ymcanyc.org
Profile Updated July 9, 2003

Evaluation 1: 1999–00 Program Implementation Report


Evaluation Description

Evaluation Purpose To assess the Virtual Y against National School-Age Care Alliance (NSACA) standards of excellence in programming, to identify features that appear to promote and impede excellence and participation in the program, and to compare program functioning across years, where data are available.
Evaluation Design Non-Experimental: Data were collected from 88 sites.
Data Collection Methods Interviews/Focus Groups: Focus groups were conducted with branch supervisors who oversee multiple Virtual Y programs. Participants were asked to identify roadblocks to quality and to propose solutions. Focus groups were conducted with parents to gather information about their perceptions of the program, staffing, and the program's influence on children's academic and psychosocial development, parent involvement, and program implementation.

Observation: Every year of the study, research staff made daylong visits to a variety of program sites to conduct observations. One site was selected for intensive analysis and was observed weekly.

Secondary Source/Data Review: Staff and child attendance records were collected monthly from October 1999 through June 2000.

Surveys/Questionnaires: Surveys were administered to Virtual Y coordinators and counselors, school principals, and parents of program participants in the spring of each program year. The surveys integrated aspects of the NSACA standards of excellence in programming.


Findings:
Formative/Process Findings

Activity Implementation In open-ended survey questions, program stakeholders (parents, staff, and school principals) acknowledged that Virtual Y children were sometimes bored and unchallenged by program activities and agreed on the importance of improving the quality of staff and program activities and on enhancing communication among program staff, parents, and teachers.

Moving forward with increasingly concrete and experiential training activities, obtaining adequate space for programming, achieving timely delivery of program supplies, and serving better tasting and more substantial snacks were also goals highlighted by program stakeholder survey data.
Recruitment/Participation Of 5,905 children who started the program in 1999–2000 (i.e., attended one day), 66% persisted throughout the program year. In 1998–1999, 65% of children participated throughout the program year.

In 1999–2000, children attended 80% of the sessions for which they were registered, compared to 73% in 1998–1999.

In 1999–2000 and earlier, children who attended the program most regularly stayed in the program the longest.

Higher principal ratings were the best predictors of higher staff and student program attendance and staff and student persistence in the program in 1998–1999 and 1999–2000. In 1999–2000, parent ratings and staff satisfaction were also good predictors.

In all three years, higher student attendance and persistence were associated with greater numbers of paraprofessionals and non-Board of Education employees on staff.

Both student attendance and persistence were lower as the number of Board of Education teachers and non-pedagogical Board of Education staff employed at the site increased.

Higher staff satisfaction was associated with lower staff turnover, higher staff attendance, and higher child attendance in 1999–2000.
Satisfaction Principals, parents, and program staff rated the program positively against NSACA quality standards-above three points on a four-point scale where 1=Never true of this program and 4=Always true of this program.
Staffing/Training The student-to-staff ratio in 1999–2000 was 12 students to one staff member, lower than the 1998–1999 ratio of 13 students to one staff member, but higher than the desired 10 students to one staff member.

A majority of staff was employed by the Board of Education concurrent with their Virtual Y position. In 1999–2000, Board of Education employees working in the Virtual Y generally felt less capable of doing their Virtual Y job than non-Board of Education employees.

The vast majority of Virtual Y staff had at least two years of college. (It is worth noting, however, that branch supervisors felt that aptitude for and experience working with children and youth were better predictors of job effectiveness than a formal education.)

Almost half of the staff spoke a language other than English; overwhelmingly this language was Spanish.

The typical age of staff members was between 20 and 29 years old.

In 1999–2000, most staff reported their ethnicity as African American or Hispanic.

Most staff members were female.

As in prior years, most staff reported satisfaction with their jobs, and they reported confidence in their abilities to do those jobs.

Twenty-one percent of counselors—those who work directly with children—found the workload heavy in 1999–2000 compared with 26% in 1998–1999. Fifty-one percent of coordinators—those who manage individual sites—found the workload heavy in 1999–2000 compared with 60% in 1998–1999.

Most Virtual Y supervisors—those who oversee multiple sites—found the workload manageable in 1999–2000. In previous years, most found the workload unmanageable.

Coordinators were absent about one day every two months and counselors were absent about one day every month, similar to 1998–1999 findings.

The annual turnover rate for coordinators is almost half of the 31% annual turnover rate for counselors, slightly higher than in 1998–1999.

Over half of the coordinators and counselors stated that they planned to return to the program in the following year, a higher percentage than in 1998–1999.

Well over half of coordinators and a third of counselors responded that they had the same job one year ago, considerably more than in 1998–1999.

Evaluation 2: Impact of the Virtual Y on Children's Classroom Behavior: Program Year 1999–2000


Evaluation Description

Evaluation Purpose To examine the impact of the Virtual Y on children's classroom behavior.
Evaluation Design Quasi-Experimental: Data on between 500 and 650 children's classroom behaviors were collected and compared during each year of the program. The research sample in each year was the group of children for whom there were parent permission for the evaluation process and both a pre- and post-program teacher rating. The group of students with valid pre- and post-program data was approximately 20% to 30% of children with parental permission for each of the three years. Given fluctuations in available data, changes in program sites, and normal student turnover, very few children (fewer than 50) were included among study participants in multiple years. The small number ruled out the possibility of meaningful longitudinal analysis.

In year 3, it was possible to assess the study sample against the full population of Virtual Y participants and nonparticipants. Compared to the larger population of the school, study children were poorer and more likely to be minorities and of limited English proficiency (LEP). They were also more likely to be female and recent immigrants. They were younger for their grade, and they had better attendance records than non-program participants.

The evaluators caution that because of these differences they could not rule out selection effects that may have accounted for pre- and post-program differences detected in their sample. To reduce the likelihood that these selection effects might bias any discovered impacts, the evaluators controlled for LEP status, gender, race, age, and prior attendance in their analyses. The regression analyses also controlled for grade-level age effects to assess the extent to which students' maturation may have been driving any discovered program effects, and maturation effects were found unlikely to have been a major factor.
Data Collection Methods Tests/Assessments: The Teacher-Child Rating Scale (Hightower, A. D., et al.) was used to obtain pretest and posttest ratings of Virtual Y participants from their classroom teachers. The instrument is composed of one scale measuring overall classroom behavior, and seven subscales measuring the following behaviors: acting out, shyness and anxiety, learning skills, frustration tolerance, assertive social skills, task orientation, and peer social skills.

References
Hightower, A. D., Work, W. C., Cowen, E. L., Lotyczewski, B. S., Spinnell, A. P., Guare, J. C., et al. (1986). The Teacher-Child Rating Scale: A brief objective measure of elementary children's school problem behaviors and competencies. School Psychology Review, 15, 393–409.
Data Collection Timeframe Data were collected at the beginning and end of each of the school years: 1997–1998 (year 1), 1998–1999 (year 2), and 1999–2000 (year 3).


Findings:
Formative/Process Findings

Activity Implementation Quality activities and enhanced program resources, such as extra books were found to be associated with better outcomes in Virtual Y programs.
Parent/Community Involvement Careful communication with both the home and host school (as rated by parents) was found to be associated with better outcomes in Virtual Y programs.
Recruitment/Participation Teachers reported a high proportion of Virtual Y participants as having behavioral problems. According to their school-day classroom teachers, 85% of participants in year 1, 91% of participants in year 2, and 89% of participants in year 3 evidenced some behavioral problem at the time of pre-program testing. The problems were most frequently reported to be in the mild range.

Across years, the highest preprogram scale scores of participants (indicating the greatest problems) were consistently in the areas of task motivation, frustration, tolerance, and learning skills.

The most consistently troubling individual behaviors for children in year 1 and year 2 were organizing themselves and functioning with distractions; for year 3, the most troubling individual behaviors were functioning well with distractions and ignoring teasing. The least troubling areas for children in year 1 and year 3 were nervous behavior and sadness. In year 2, the least troubling areas for children were nervous behavior and overly aggressive behavior towards peers (fighting).
Staffing/Training Regression analysis identified the following features of the Virtual Y associated with better outcomes: positive student and staff relationships; staff perception that their workload is manageable; higher percentages of staff who are non-Board of Education employees; staff who rely especially on YMCA trainings in developing their lessons; enhanced program resources such as extra staff; higher percentages of staff with four-year college degrees; lower student to staff ratios; staff who speak more than one language; higher percentages of staff who make use of fellow counselors' suggestions; and higher percentages of staff who are teachers.


Summative/Outcome Findings

Academic In all years, the data showed statistically significant (p<. 01) and moderate to large improvements on the learning skills subscale.
Youth Development In all years, the data showed statistically significant (p<. 01) and moderate to large improvements on all seven subscales (acting out, shyness and anxiety, frustration tolerance, assertive social skills, task orientation, and peer social skills) and on the overall behavior scale. The effects were found for children at each level of behavioral difficulty, from mild to severe.

Evaluation 3: Preliminary Analysis of Virtual Y After-School Program Participants' Patterns of School Attendance and Academic Performance, Final Evaluation Report, Program Year 1999–2000


Evaluation Description

Evaluation Purpose To examine the effect of the Virtual Y program on participants' academic progress.
Evaluation Design Quasi-Experimental: Virtual Y students were compared with nonparticipants in the same grade levels at the same schools. Only Virtual Y participants who attended the program for at least 49 days, who had not been left back the prior year, and who were not in Project Read (the Board of Education's supplemental reading program) were included in the program group. Comparison group students were also not enrolled in Project Read. Statistical comparisons between these groups showed significant preprogram differences. These were controlled for in all analyses.
Data Collection Methods Secondary Source/Data Review: Student attendance and standardized reading and math test data were obtained.
Data Collection Timeframe Data were collected for school years 1998–1999 and 1999–2000.


Findings:
Formative/Process Findings

Staffing/Training Hierarchical regression analysis showed two Virtual Y program elements to be factors associated with improved student outcomes in reading, math, attendance, and (based on data from a prior study in this series) classroom behavior: hiring staff with four-year degrees and maintaining 10:1 student to staff ratios.

An additional four program elements were found to be associated with improvements in three of the four outcome areas: hiring staff under 30 years old (not associated with improvements in classroom behavior), maintaining low coordinator turnover rates (not associated with improvements in classroom behavior), ensuring staff were trained in youth development practices and drew on lessons in program manuals (not associated with improvements in reading), and hiring Board of Education teachers (not associated with improvements in reading).


Summative/Outcome Findings

Academic The average school attendance of third- and fourth-grade children participating in the Virtual Y exceeded the average attendance of children in the comparison group taking into account initial differences in student attendance and demographic background (p<.05 and p<.01, respectively). The difference between the mean school attendance performance of second grade Virtual Y students and comparison group students was positive, but not statistically significant.

Appropriate data on reading were available only for fourth grade students. Means comparisons, taking into account students' demographic features, prior reading skill, and school attendance, showed that post-program differences in reading skill between children in the two groups were not statistically significant.

Appropriate data on math were again available only for fourth grade students. Means comparisons, taking into account students' demographic features, prior math skill, and school attendance, showed that post-program differences in math skill between children in the two groups were significant (p<. 01). Children participating in the Virtual Y evidenced better performance on citywide math tests than children in the comparison group.

The foregoing analyses may have underestimated the impact of after school programming as students in the comparison group were likely to have been involved in alternative after school programs (programs other than the Virtual Y). Regression studies were conducted to understand the value that Virtual Y added to student performance. These studies showed that the program contributed to improved school attendance, reading skills, and math skills.

© 2016 Presidents and Fellows of Harvard College
Published by Harvard Family Research Project