You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.

The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.

Terms of Use ▼

Anita Baker and Constancia Warren of the Academy for Educational Development describe their evaluation of New Jersey's School-Based Youth Services Program.

In 1987, the New Jersey Department of Human Services created the School-Based Youth Services Program (SBYSP) with an allocation of $6 million. This funded the establishment of the first statewide initiative in the country to integrate a range of services for adolescents in one location at or near schools. These projects began their first full year of operation in 1988, in 29 New Jersey communities in 22 local high schools, two regional high schools, and five vocational-technical schools. There is at least one project in every county of the state, and all students at host schools can participate in SBYSP activities. Each project is managed by a lead agency, and a 25 percent local resource match is needed to secure funding.

Core school-based services include individual and family counseling; primary and preventive health services (on site or through referral); employment counseling, training, and placement; drug and alcohol abuse counseling; and recreation. Local programs enjoy substantial flexibility in how they provide services and activities in response to local needs and resources. The desired impacts of the program include improved educational and health outcomes and better economic prospects for youth, as well as reduced need for more intensive (and expensive) services. The work of local projects is enhanced by a central support team in the state Department of Human Services that links projects to a network of local providers and additional resources; this team has played a critical role in the longevity and vitality of the SBYSP projects.

Evaluation Design

Supported by the Annie E. Casey Foundation, the Academy for Educational Development (AED), a nonprofit educational evaluation and technical assistance organization, is conducting an evaluation of SBYSP that will run for 3 and 1/2 years.

Phase I. During the first phase of the evaluation, AED completed an analysis of SBYSP's evolving policy context. In addition, following visits to all sites and interviews with key project and school staff and youth, AED completed a cross-site analysis of project implementation. Data gathered during this phase were also used to articulate a theory of action in each program content-area and design a longitudinal study of student outcomes in six of the sites. SBYSP project directors helped develop both the theory of action and the outcome study design.

Phase II. During the second phase, AED is following the 1996–97 incoming ninth-grade cohort into its junior year (fall 1998) to measure outcomes for individual youth. This will entail periodic surveys of the entire cohort, review of SBYSP participation/utilization and school data, and case studies of “typical” youth at each site. The evaluators will compare SBYSP participants to nonparticipants with regard to a range of personal attitudes, situational stresses and assets, and behavioral outcomes in health, mental health, substance abuse, employment preparation, educational progress, and positive youth development.

Some of the challenges the evaluators face include:

  • Understanding the dynamic relationship between services and outcomes in a multiservice program. In such programs, the total “package” of services and activities varies from student to student. This complex task is rendered more difficult by variations among sites. For example, health services in one site involve referrals to local providers and assistance with transportation and scheduling, while in another, physical and psychosocial screening is provided on site, reinforced by referral to a local provider, and in a third project, primary care services are provided on school grounds.

  • Developing instruments that are comprehensive and sensitive, yet feasible to administer. Measuring outcomes related to the full range of needs addressed and program services and activities constitutes a challenge endemic to the evaluation of multiservice, comprehensive, or service-integration projects. Previous researchers and evaluators of single-focus studies, upon which most of our current instrumentation is built, have been able to devote entire instruments to individual or subsets of outcome indicators. However, the length and complexity of the instruments used are constrained by the class period during which a survey can be administered, since it is very difficult to mount multiple survey administrations during regular class time in a single school year. To make effective use of limited time, it is critical to remain in close contact with the practitioners concerning evaluation priorities and to use their theory of action to inform the instrument and study design.

  • Drawing comparisons that are sensitive to the factors that affect outcomes. Given the degree of variation among the six case-study sites and the likelihood that SBYSP users have more severe problems than nonusers, simple comparisons between users and nonusers are inappropriate. Instead, AED is using key variables on the baseline survey to select those nonparticipants who resemble program users based on a broad set of characteristics, including demographic (gender, race/ethnicity); situational (family stress, outside sources of support); personal (depression, optimism, aspirations); and preexisting behavioral characteristics (community service, academic misbehavior).

Constancia Warren
Senior Program Officer

Anita Baker
Senior Technical Consultant

Academy for Educational Development
100 Fifth Avenue
New York, NY 10011
Tel: 212-243-1110
Fax: 212-627-0407

‹ Previous Article | Table of Contents | Next Article ›

© 2016 Presidents and Fellows of Harvard College
Published by Harvard Family Research Project