The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.
Volume VI, Number 1, Fall 2000
Issue Topic: Out-of-School Time Issue #1
From the Director's Desk
 |
| Heather Weiss |
As the working parent of a teenage daughter, I have been aware of the importance of safe, fun, and challenging after school activities for quite some time. I am pleased, therefore, that a consensus is emerging about the importance of after school programs—both in policy circles and among the public. There is tremendous activity in the after school field, evidenced by the variety of organizations becoming involved in designing and administering programs and the substantial recent growth in public and private funding. From this activity, innovative partnerships and community-based collaboratives have formed and significant investments in evaluation and research are increasingly being made. The next few years will be critical as those who are invested in the field of after school—policymakers, practitioners, funders, evaluators, researchers, parents, children, and youth—build on the current momentum and create high quality, sustainable after school programs for everyone who needs them.
Over the past year and with support from the Charles Stewart Mott Foundation, HFRP “mapped” the after school field. We identified the key stakeholders, talked with them in focus groups, one-on-one interviews and meetings, and gathered a variety of perspectives as we tried to identify the critical issues and opportunities facing the field. From this work, several challenges emerge that have important implications for evaluation and its contribution to the development of the after school field.
A substantial challenge that confronts many programs that serve families and children, including those in after school, relates to how to respond to the demand for outcomes measurement in the current era of accountability. After school programs increasingly must show results at the national, state, and community levels, creating an atmosphere that has changed how stakeholders need to think about the evaluation of their programs.
Evaluations of large-scale initiatives will continue to provide, as in the past, crucial information regarding the effectiveness of large public and private investments. Equally important are local evaluations that help local programs make their case to state and community stakeholders. The new landscape of accountability dictates that local programs need a range of supports to build capacity to identify and measure results in ways that are both manageable and cost effective.
A second challenge is ensuring that information gathered at the local and national levels is widely available and used. Gathering and reporting data to demonstrate accountability is important, but information should also be used to make improvements to programs and policies. In order to continually improve, evaluative information must be relevant, particularly to program stakeholders who need critical feedback to guide program improvements and long-term plans.
More and more, programs are expected to demonstrate their success through measuring program outcomes, making the challenge of determining appropriate outcomes for evaluation measurement more critical than ever. Currently, programs within the field have outcomes ranging from improving academic achievement and developing more effective juvenile justice to strengthening athletic and physical ability and promoting life skills, with companion goals and objectives. Program expectations should be tied to the kinds of activities they offer, and funder investments in evaluations need to reflect the wide variety of programs in the field. Like the other challenges, determining the key outcomes for after school programs requires continuing conversations and broad access to evaluation results among the many stakeholders in the field.
Finally, we are in the early stages of developing theories linking interventions and outcomes. As this work progresses and more evaluations become available, stakeholders will begin to come to consensus about what kinds of programs and activities have the best chance of achieving positive outcomes for children in after school programs. Bringing the developmental research theories and findings together with information from program evaluations to inform long-term planning, decision making, and day-to-day activities of programs is critical as we lay a solid foundation on which to build the field.
In many ways, these four challenges—building evaluation capacity, incorporating continuous improvement as a component of good practice, selecting appropriate outcomes, and strengthening the connection between outcomes and theory—are not unique to after school, but they are particularly urgent for the field of out-of-school time. This is a fast-growing period in which there is a reassessment of the role of evaluation vis-a-vis social programs and policies. The work going on in after school programs can both stimulate and benefit from the debates in other fields about the purposes, audiences, and methods of evaluation.
In order to address these four challenges, we must think innovatively and creatively. Our focus group discussions point to several priorities: Among these are the need to build capacity not only to collect information and conduct sound studies but also to make productive use of information. This is basic to the field of evaluation (capacity building is, in fact, the theme of this year’s American Evaluation Association conference.
We need to continue ongoing dialogue among stakeholders at a variety of levels. We need discussions among evaluators from both large- and small-scale studies to share ideas about issues that affect evaluation in the after school field—those identified here and others that will emerge over time. Conversations among evaluators, researchers, policymakers, funders, and practitioners are also important to ensure that evaluation addresses pertinent policy and practice questions.
Finally, we think these challenges require the development of an infrastructure to support the important work done not only by after school programs, but also by those who evaluate them and make policy that affects them. We talk more about this in the Theory & Practice section of this issue.
In keeping with this newsletter’s goal to be open, accessible, and serve its readers, several format changes debut in this issue. The newsletter has a new look, thanks to HFRP’s new publications team: Christine Phelan, communications manager, and Leslie Goodyear, editor of The Evaluation Exchange (the photo of me was their idea, not mine!). We’ve added a new section, Ask the Expert, and merged New & Noteworthy and Electronic Resources into a single column. These changes are in response to suggestions from the reader survey we conducted last year. We continue to make better use of electronic media—our website is continuously updated and now includes a section specifically on our after school work.
In the spirit of bringing together multiple perspectives in the after school field, we have increased our mailing list for this issue. To new readers, I bid a warm welcome. If you wish to receive a free subscription to The Evaluation Exchange, visit our website, or contact us directly. We strive, above all, to be an interactive newsletter examining timely and innovative issues in evaluation, research, and continuous improvement. We always welcome suggestions and comments—whether you are a first time reader or a regular one.
Heather B. Weiss, Ed.D.
Founder & Director
Harvard Family Research Project