Volume X, Number 4, Winter 2004/2005

Issue Topic: Evaluating Family Involvement Programs

Spotlight

Basic Applications of Multilevel Modeling

Eric Dearing, from the University of Wyoming, explains some of the basic uses of multilevel modeling, using examples from family involvement research and evaluation.

Multilevel modeling1 has become an increasingly popular means of analyzing data. A variety of software programs may be used to estimate multilevel models, and recent books have made this analytic tool highly accessible.2 Nonetheless, a communication gap often exists between researchers using multilevel models and consumers of their work with less expertise in this area. This gap likely interferes with the translation of research into practice. Using recent research examples, this article briefly addresses some basic uses of multilevel analysis to orient readers who have little familiarity with this method.

One stumbling block to communicating results from multilevel models has been the use of several terms to describe the same modeling technique, terms such as multilevel, hierarchical linear, mixed-effects, and random-effects. In evaluation research, multilevel modeling and hierarchical linear modeling are common terms, primarily because they identify a central feature of the analyses: nested data. Nested data refers to data that are arranged in a hierarchical, multiple-level structure, for example, data in which multiple children attend the same school (i.e., children are nested within schools) or longitudinal data in which there are multiple observations of the same children (i.e., observations are nested within children).

When conventional analytic methods not designed for multilevel analyses are being used, nested data create both statistical and conceptual problems. Conceptually, for example, questions may arise about the appropriate unit of analysis. In the case of children nested within schools, a researcher may wonder whether children, or schools, should be the unit of analysis. Using multilevel models, researchers may simultaneously estimate patterns of association at child and school levels, or, in longitudinal data, simultaneously estimate patterns of stability and change within children and variations across children.

Clements, Reynolds, and Hickey,3 for example, used multilevel models to examine predictors of verbal performance in a sample of 1,539 children who attended 25 schools (or other early education sites)4 providing educational and family-support services. Analyses were estimated in two levels: Child and family predictors were estimated at the first level, and site predictors were estimated at the second level. For example, the association between family risk (a composite of factors, including parent education) and performance was estimated at the first level of analysis; the association between average parental involvement at sites and performance was estimated at the second level of analysis.

Based on the first level of analysis, the authors report a significant and negative effect for family risk, indicating that, on average, risk was associated with lower verbal performance. The authors also report that the size of this association varied significantly across schools. In the second level of their analysis, the authors examined school characteristics associated with variation in children’s verbal performance. Children at schools with high levels of parent involvement displayed higher levels of verbal performance than children at other schools. Thus, considering both levels of analysis, Clements et al. simultaneously examined child and school predictors of verbal performance.

Dearing, McCartney, Weiss, Kreider, and Simpkins5 used multilevel models to examine associations between family involvement in education during kindergarten and children’s literacy performance from kindergarten through fifth grade. In the first level of their multilevel model, average rate of change in literacy performance and the extent to which children varied in this rate of change were examined. On average, literacy performance increased over time, yet increases were greater for some children than others. In the second level of their analysis, the authors examined whether level of family involvement at kindergarten helped explain variations across children with regard to rate of change in literacy performance. They report that higher levels of family involvement were associated with greater gains in literacy achievement, particularly for children whose mothers were less educated. Considering both levels of analysis, Dearing et al. were able to examine average changes in literacy performance as well as between-child differences in literacy performance.

Given their usefulness, it is likely that multilevel models will become an increasingly popular means of analyzing data relevant for science, practice, and policy. These two examples provide only portions of the analyses completed in the respective studies and highlight only some of the uses of multilevel modeling. Nonetheless, they illustrate some of the basic capabilities and purposes of multilevel models and, as such, provide a useful starting point for those interested in results generated with these methods.

1 Multilevel modeling refers to the simultaneous analysis of hierarchically arranged data.
2 Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data analysis: Modeling change and event occurrence. New York: Oxford University Press.
3 Clements, M. A., Reynolds, A. J., & Hickey, E. (2004). Site-level predictors of children’s school and social competence in the Chicago Child-Parent Centers. Early Childhood Research Quarterly, 19, 273–296.
4 Not all of the sites in the Clements et al. study were in schools, but for purposes of brevity and clarity, I use “schools” to refer to these early education sites.
5 Dearing, E., McCartney, K., Weiss, H. B., Kreider, H., & Simpkins, S. (2004). The promotive effects of family educational involvement for low-income children’s literacy. Journal of School Psychology, 42(6), 445–460.

Eric Dearing, Ph.D.
Assistant Professor of Psychology
Department of Psychology
University of Wyoming
Laramie, WY 82070
Tel: 307-766-6149
Email: dearinger@uwyo.edu