The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.
Volume IV, Number 1, 1998
Issue Topic: Performance Measurement
Evaluations to Watch
Ensuring Quality in Head Start: The FACES Study
Ruth Hubbell McKey of Ellswood Associates and Louisa Tarullo of the U.S. Department of Health and Human Services discuss the FACES study of Head Start.
As the nation's premier early childhood education program, Head Start is leading the way in developing and reporting on its accountability for comprehensive services to approximately 800,000 children and their families each year. Head Start has developed an outcome-oriented accountability system, which combines scientific research with program-level reporting and monitoring, and is based on a consensus-driven set of criteria for program accountability.
The Head Start Program Performance Measures Initiative is a response to a specific legislative mandate, strategic planning for Head Start, and broader public emphasis on accountability and results-oriented evaluation. Measures were developed in accordance with the recommendations of the Advisory Committee on Head Start Quality and Expansion, the Head Start Act, and the Government Performance and Results Act.
The Conceptual Framework
In 1995, Head Start undertook a consensus-building process involving Head Start program staff and parents, early childhood organization representatives, researchers, experts in the education, child development and early intervention fields, and federal officials. A conceptual framework was developed to display the linkages between process and outcome measures for Head Start children and families. The ultimate goal of Head Start is to promote the social competence of children, defined as the child's everyday effectiveness in dealing with his or her present environment and later responsibilities in school and life. Social competence is depicted at the top of a pyramid, with five objectives supporting it:
- Enhance children's healthy growth and development.
- Strengthen families as the primary nurturers of their children.
- Provide children with educational, health, and nutritional services.
- Link children and families to needed community services.
- Ensure well-managed programs that involve parents in decision making.
An important aspect of the pyramid is the strong empirical connection between the provision of quality services (process measures) and improvements in child development (outcome measures). Each Program Performance Measure has performance indicators that specify how the measure will be assessed. For example, the objective “Enhancing children's healthy growth and development” includes as one performance measure: “Head Start children demonstrate improved emergent literacy, numeracy, and language skills.” The performance indicator for this measure is the change in the Head Start children's emergent literacy, numeracy, and language skills over the Head Start year, measured by individual child assessments and parent and teacher reports of the child's abilities. A more process-oriented measure is: “Head Start assures that children receive needed medical, dental, and mental health services.” The performance indicator for this measure is the number and percent of Head Start children who received needed medical services as reported by the programs themselves. In order to provide annual progress reports on the indicators supporting each of the objectives, data will be drawn from agency level sources, such as the Head Start Program Information Report (PIR)—a program-level reporting system completed by each Head Start program annually—and program monitoring reports, as well as from the classroom, teacher, family, and child level.
The Outcome Study
Most of the outcome data are being collected from the Head Start Family and Child Experiences Survey (FACES), a longitudinal study of a nationally representative sample of Head Start programs, classrooms, teachers, parents, and children examining the quality and effects of Head Start. FACES is being conducted by a team of researchers under contract to the Research, Demonstration and Evaluation Branch, Administration on Children, Youth and Families, US Department of Health and Human Services.
The Spring 1997 field test of FACES on a national probability sample of 2400 children and families in 40 programs collected extensive information about the quality of the educational services using classroom observations and interviews with teachers and administrators. FACES measured the abilities of Head Start children on a variety of child development constructs using direct child assessment, observation, and parent and teacher reports. These measures, often with nationally normed comparison data available, tapped abilities in emergent literacy, numeracy, and language skills; general cognitive skills, gross and fine motor skills; attitudes toward learning; social behavior and emotional well-being; and physical health. In addition, parents were interviewed about their life experiences, and involvement and satisfaction with Head Start. Classrooms were assessed on scheduling, the early childhood learning environment, and teacher behavior. Research has consistently linked program quality variables such as child:adult ratio, group size, responsiveness of teacher-child interaction, and richness of learning environments to improved child outcomes. For the first time using a national sample, FACES is testing the same linkages in Head Start. It is expected that higher quality Head Start programs will have children showing higher levels of skills and, over time, displaying greater gains in developmental outcomes.
Next Steps
Because of its unique opportunity to collect longitudinal child outcome data tied to program quality, FACES continued with Fall 1997 and Spring 1998 data collections on a nationally representative sample of 3,200 children and families in the same 40 programs. These phases will allow for a pre-post comparison, assessing the effects of Head Start by examining children and parents before their exposure to Head Start and determining their status at the end of the program. The Spring 1998 data collection will also examine a cohort of former Head Start children from the original field test sample who are completing kindergarten, to determine how they have performed in their first year of school. A Spring 1999 FACES data collection will examine results after kindergarten for the Fall 1997 Head Start cohort. Analyses of the Fall 1997-Spring 1998 comparisons will be available in late 1998.
Additional efforts to improve the quality and effectiveness of Head Start are also underway. A consortium of four Quality Research Centers (QRCs) was established in 1995 to collaborate with local Head Start programs and the federal Head Start Bureau to define, assess, and verify the effectiveness of high-quality program practices in Head Start programs. On January 1, 1998, revised Head Start Program Performance Standards were implemented. These regulations delineate the operations and quality of services to be offered by all Head Start programs. Concurrently, the Head Start program monitoring system is being revised to streamline the process, and promote greater consistency and reliability across HHS regions. The training and technical assistance system has also been recently redesigned.
Performance Measures Progress Reports are available on the Head Start Bureau Web page: www.acf.dhhs.gov/programs/hsb or can be requested by fax: 703-683-5769; or email: hspmc6@mail.idt.net
The evaluators for the Head Start FACES Study are Nicholas Zill and Gary Resnick of Westat, Inc.; David Connell of Abt Associates; Ruth Hubbell McKey of Ellsworth Associates; Robert O'Brien of The CDM Group. Federal project officers are Louisa B. Tarullo and Henry M. Doan of the Research, Demonstration and Evaluation Branch, Administration on Children, Youth and Families, US Department of Health and Human Services.
Ruth Hubbell McKey
Vice President for Research and Program Services
Ellsworth Associates, Inc.
Louisa B. Tarullo
Research, Demonstration and Evaluation Branch
Administration on Children, Youth and Families
US Department of Health and Human Services
Email: lbtarullo@acf.dhhs.gov