The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.
Volume VII, Number 2, Spring 2001
Issue Topic: Out-of-School Time Issue #2
Theory & Practice
Supporting Effective After School Programs: The Contribution of Developmental Research
In this article, Jacquelynne Eccles, McKeachie Collegiate Professor of Psychology, Women’s Studies, and Education, and editor of the soon-to-be released volume on community based programs for youth, sponsored by the National Research Council in conjunction with the National Academy of Science, shares her thoughts about the contribution of developmental research to the after school conversation and the need for an infrastructure to support this.
After school programs are everywhere: in schools, community centers, parks, and youth serving organizations. Increased concern on the part of parents, educators, researchers, and policy makers about the kinds of activities youth are involved in during out-of-school time has led to this recent growth spurt for after school programs. The challenge facing these programs is to provide healthy, developmentally appropriate, and enriching experiences for youth, while at the same time meeting demands to show the resulting benefits that accrue from participation. In concert with practitioners, educators, parents and policy makers, developmental researchers can help to identify the pathways that lead from fun activities to positive outcomes for youth.
The Youth Development Field Today
Developmental research about adolescence has changed tremendously in the twenty-five years since it first came on the research radar screen. Always a hard developmental period to study because of the many and interrelated changes going on in young people’s bodies and lives, we now understand that longitudinal, highly contextualized, and inter-disciplinary research are the most effective approaches for studying this age group.
In the years since adolescence became a focal point for developmental researchers, we have seen our perspective within the field evolve from thinking of adolescence as a problem and conflict-ridden period to one in which positive youth development ideals are promoted. This refocusing on positive youth development has come about as a result of the work of developmental researchers, largely supported through early money targeting prevention activities. It is supported by the strong practice-based knowledge of large organizations such as Boys Scouts, Girl Scouts, and the Y’s as well as countless other, much smaller programs helping kids. These programs have a broad-based goal to help children and adolescents develop across a wide array of areas of their lives—skills areas, social areas, emotional areas, moral areas, and citizenship areas. These strong advocates from the world of practice, in concert with researchers and policy makers, have been a voice for youth for many decades and have brought us, in large measure, to the place we are now.
Some describe the current youth development field as divided between those who promote prevention and those who advocate positive youth development; although there is this tension, I don’t believe it is necessary. From my perspective, both the prevention perspective and the positive development perspective have contributed to the excellent work done on the key ingredients for successful adolescence—ingredients that we all agree contribute to healthy, well-functioning youth. Both these perspectives have their place in the development, implementation and evaluation of youth-serving organizations and after school programs.
The more important question, it seems, is whether current investments in after school programs are going to focus primarily on educational remediation or more general, positive development. I think this question is being answered in the field. If you look at after school programs on the ground, most are far more than just tutorial programs. People are interested in remediation, but they’re also very interested in good places for kids to be. And they know youth will walk out of those places that are just tutorial. So we see programs emerging that are very broad and comprehensive. And this, I think, is very good.
The Youth Development Field and Accountability
There is no question that the current interest and investment in after school and other youth programs often comes with heavy strings attached—accountability demands. While accountability has always been an issue for organizations and programs that serve youth, the nature of this accountability has changed. Today, individual programs have to show that what they are doing is producing important youth outcomes. This is very different from the more general question, “do we know in principle that this intervention works”, that dominated the field of program evaluation in the past.
In recent situations, randomized trials can and have been used to make the case for success. The Public/Private Ventures (P/PV) study of Big Brothers/Big Sisters is one example; P/PV showed that mentoring can make a difference and it is an activity that can occur in lots of organizational settings. There were programs like Big Brothers/Big Sisters that were shown to be successful through these research methods, but there were also lots of program evaluation failures due to the difficulties of doing high quality randomized trial studies in this field. Often the programs were evaluated before they had been well-implemented. In addition, the voluntary nature of participation in such programs makes controlling the exposure to the program very difficult. Consequently even large-scale, well-designed randomized trials often ended up showing very small effects.
There is a need to think about what evidence is most compelling and necessary in an era when individual programs must show that they produce important results for children and youth. Even though randomized trials are the most scientifically rigorous way to demonstrate this, such studies are often not feasible. And the results from such studies, when they are not done appropriately, can be very misleading.
Whether these types of experimental studies help identify the best programs for youth is also a compelling question. Based on the work of Milbrey McLaughlin at Stanford University, it follows developmental theory perfectly that programs that are likely to be most attractive to youth—and to have the most positive results for youth—are programs that change over time, that accommodate the kids, and that are co-constructed by the kids and the adults in that setting. Programs like these are also heavily dependent upon charismatic individuals who care a lot about the kids. Elements that make programs great, such as charismatic staff, are impossible to study with a randomized design since the “it” is not clear and unchanging; the “it” is really an evolving relationship between all the participants in the program.
The Role of Developmental Research
So, where does that leave an organization that has to be accountable? I think program staff need to be very thoughtful about what they think they’re affecting. They need to lay out the theory that underlies their programs and they’ve got to measure and monitor all of the aspects of their program they believe are important over time, along with carefully measuring the adolescent “outcomes” they think their program is affecting.
The theory they lay out could come from various places. It could be based on other models of similar programs. It could be organic, deriving from the thinking and unique approaches or populations of the program itself. It could come from research.
This is where collaboration among developmental researchers and practitioners can be so productive. If a program is hoping to demonstrate that it did something, it has to have a very solid theory behind it. The field is young enough that we’re still working out the models of change. We need to consider and evaluate as many of these models as possible. We need to be thinking broadly about what experiences are important for youth, and why. What outcomes would come from those experiences, and how can we measure those outcomes? The next step is to put these models and measures in place for programs.
This exchange could play out in several ways. Program people could identify the things they would like to give youth: “well, I think it’d be nice to give my kids X, Y, and Z experiences, because ... these experiences ought to produce change on X, Y, and Z indicators of well-being.” Or, “I would like my kids to be A, B, C, and D. How do I get them there?”
Another important piece in this process includes deliberating and reflecting on information gathered as a means of continuous improvement. Reflective practice means that organizations are constantly measuring what they are providing as well as what they think they will affect. It is important to realize that programs are continually changing; they are not static. It is not possible to evaluate them, show they work and consider it done. It is important to invest in putting into place a whole system of systematic monitoring. Documenting everything from who participates and who doesn’t to what participation means and what happens to those who participate.
The other challenge, or course, is that of determining the most appropriate level of expected and measured outcomes. A program that only deals with 100 kids should not be responsible for moving the whole community. If communities want to move massively, they need to think at the community level about the array of programs that are available for their youth. At the community level, it seems the model should be working together to put in place a wide range of programs that will meet those needs, holding the individual programs accountable for reflective practice, and then, monitoring at the community level to determine if trends change over time.
Support Through Systems
We, as developmental researchers, evaluators, program practitioners, and stakeholders interested in youth-serving programs, need to think systemically and creatively about how to best support this. This system would insist that people who have the research expertise participate as partners in the creation of the larger of these model programs. The hope is that these models will then filter down to the more “mom and pop” small community programs. It is important, too, to create good, easy-to-administer measures and provide workshops and on-the-job training for those who administer programs.
The model of the Land Grant colleges is a wonderful model of research-practice collaboration. The extension programs in such colleges provide staff who go out to communities and help communities think about their needs and about the type of research needed to find answers that will help the community solve their problems. There’s no reason that Land Grant colleges and other colleges can’t follow this model of university-community collaboration to help in the youth development field. There are some that are, through their 4-H outreach programs and other cooperative extension programs—for example Steve Small at the University of Wisconsin and Dale Blyth at the University of Minnesota. Other non-profit organizations such as The Search Institute and Public/Private Ventures and researchers and practitioners like James Connell and Michelle Gambone, Karen Pittman, and Shirley Brice-Heath are working on models of positive youth development through research-practice collaboratives.
I am very excited about the emergence of such research-practice collaborative efforts. We are at a unique juncture in our deliberations about out-of-school time. Politicians, practitioners, educators, employers, academics, parents, and citizens have all become interested in youth and how to best provide them with the experiences they need to have a productive and safe adolescence as well as to grow into productive, responsible adult citizens. The time is right to bring these groups together to design and then study the kinds of programs that will provide these experiences.
Jacquelynne Eccles, Ph.D.
University of Michigan
IRWG, 1251 Lane Hall
204 S. State Street
Ann Arbor, MI 48109
jeccles@umich.edu