You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.

www.HFRP.org

The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.

Terms of Use ▼


As promised in our inaugural issue, our section, “Questions & Answers,” features responses from experienced evaluators to commonly-asked evaluation questions.

Recent evaluation results suggest that school-linked services benefit the children and families they serve most intensively. Findings from SRI's evaluation of California's Healthy Start school-linked services initiative in 40 California communities (Wagner, et al., 1994) and Philliber Research Associates' evaluation of Walbridge Caring Communities in St. Louis (Philliber Research Associates, 1994) indicate the following.

School Performance
In both the Healthy Start and Walbridge evaluations, grades and classroom behaviors improved for students who were intensively served. For example, Walbridge students who received intensive services had a 27% increase over time in teacher ratings of their work habits, a 16% improvement in ratings of their social-emotional growth, and a 23% improvement in grade point average. An analysis of Healthy Start children who received intensive services showed a significant improvement in grade point average, particularly for younger students and for those who were performing least well before participating in Healthy Start. Teacher ratings of student behavior also improved significantly for intensively served Healthy Start students (from an average scale score of 1.54 to 1.75, p<.01).

Launched in 1991 by Senate Bill 620, Healthy Start aims to restructure California's education, health, mental health, and social service systems for children and families. Its central strategy is the use of state- and local-level collaboration. In 1992, 40 local collaboratives received three-year grants to implement comprehensive, integrated, school-linked services. These funds have gone to schools of different sizes, with different populations, and in different geographic areas. Program designs vary according to community need and local resource availability. All, however, serve populations with “serious and multiple” needs (Wagner, et al., A Healthy Start for California's Children and Families: Early Findings from a Statewide Evaluation of School-Linked Services, Menlo Park, CA: SRI International, June 1994).

Caring Communities is a pilot program at Walbridge Elementary School in St. Louis, Missouri which provides integrated services to children and their families at the school site. Collaborative partners include the Danforth Foundation, the St. Louis Public School District, and Missouri's Departments of Elementary and Secondary Education, Mental Health, Social Services, and Health.

Basic Needs
After six months of participation in Healthy Start services, there were significant reductions among intensively served Healthy Start families in their reported needs for food or clothing (from 32% to 20% needing food and from 30% to 20% needing clothing, p<.001 and .05), as well as emergency funds, transportation, and childcare. Similarly, parents at Walbridge Caring Communities were significantly more likely than parents at an adjoining school to report that the school “helps a lot” with food, clothing, or other basic needs (41% vs. 13%, p<.001).

Health Care Experiences
For intensively served Healthy Start clients, there has been an increase in participation in California's Child Health and Disability Prevention program (from 19% of families to 26% having children in the program, p<.05) and a reduction in health care due to illness or injury (from 36% to 29%, p<.001). Also, there was a reduction in reported problems with gaining access to medical and dental care (from 41% to 29% and from 55% to 41% respectively, p<.001). Not only did Walbridge parents report fewer problems with health care access, they were more likely than parents at a comparison school to report that it is “easy” for students to get help with health problems (96% vs. 59%, p<.001) and that the school “helps a lot” with their own health care needs (47% vs. 25%, p<.01).

 

Further Reading

Wagner, M., Golan, S., Shaver, D., Newman, L., Wechsler, M., & Kelley, L. (1994). A healthy start for California's children and families: Early findings from a statewide evaluation of school-linked services. Menlo Park, CA: SRI International.

Philliber Research Associates. (1994). An evaluation of the Caring Communities Program at Walbridge Elementary Schooll. Accord, NY: Author.

Emotional Health
Indicators of emotional health among Healthy Start core families show reduced rates of reported depression (28% to 22%, p<.01) and suicidal thoughts (7% to 3%, p<.01), and fewer problems with hostility and anger (23% to 19%, p<.05). Walbridge parents were significantly more likely than parents at an adjoining school to report that it was “easy” for students to get help with behavior problems or family issues.

More complex behaviors associated with family functioning did not show significant change in either Healthy Start or Caring Communities families. Participation of youth in several risk behaviors, including drug and alcohol use, were unchanged after six months of Healthy Start involvement, and the Walbridge evaluation did not detect the hoped-for reduction in student involvement with social services or juvenile justice systems. Impacts on outcomes such as these will need to be a focus of longer-term evaluations.

Mary Wagner, Ph.D.
Program Manager, Educational and Human Services Research
SRI International


There are many challenges to producing evidence of the effectiveness of school-linked service programs. Traditional evaluation strategies tend to alienate program participants—particularly families—and many school-linked service programs suffer from high participation attrition rates (Wang, Haertel, & Walberg, 1994). Since family involvement and parent participation are vital for program delivery and effectiveness (White, Taylor, & Moss, 1992), high attrition rates hinder an evaluator's ability to assess a program accurately. Or, they may lead that researcher to conclude that since the program has a high dropout rate, it must be ineffective. Yet, there may be reasons for attrition other than program design, and evaluators need to be careful to determine what those may be.

 

Further Reading

Gomby, D. S., & Larson, C. S. (1992). Evaluation of school-linked services. The Future of Children, 2, 68-84.

Levy, J. F., & Copple, C. (1989). Joining forces: A report from the first year. Alexandria, VA: National Association of State Boards of Education.

White, K. R., Taylor, J. J., & Moss, V. D. (1992). Does research support claims about the benefits of involving parents in early intervention programs? Review of Educational Research, 62, 92-125.

Wang, M. C., Haertel, G. D., & Walberg, H. J. (1994). The effectiveness of collaborative school-linked services. Philadelphia, PA: Temple University, The National Center on Education in the Inner Cities.

Outcomes that are commonly addressed in evaluations of school-linked service programs include:

  • school attendance,
  • achievement scores,
  • grades,
  • behavioral problems,
  • drop-out rates, and
  • self-esteem.

Overall, there is documentation of positive results of school-linked programs on these student outcome measures (Gomby & Larson, 1992; Wang, Haertel & Walberg, 1994). Programs that appear to be most effective concern (1) pregnancy prevention and parenting, (2) parent education and school readiness, and (3) substance abuse prevention (Gomby & Larson, 1992). These program areas are more likely to be most effective in preventing school attrition and in improving attendance and self-esteem. Many evaluators, however, treat these results with caution since they are often not based on traditionally rigorous evaluation methodology.

Most of the program evaluations on which these data are based have concentrated primarily on measuring program outcomes. Little attention has been given to process and implementation data. Yet, process data is essential in documenting and validating the effectiveness of school-linked interventions.

James Davis, Ph.D.
Professor, Department of Educational Studies
University of Delaware

‹ Previous Article | Table of Contents | Next Article ›

© 2016 Presidents and Fellows of Harvard College
Published by Harvard Family Research Project