The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.
Volume IX, Number 1, Spring 2003
Issue Topic: Evaluating Out-of-School Time
From the Director's Desk
 |
| Heather Weiss |
As families face increasing pressures to find safe, productive places for their children to be during the nonschool hours, the need for high quality out-of-school time programs continues to expand. While investments in out-of-school time programs are rising in response to this demand, they do so against a heightened backdrop of accountability—in which programs are being asked to ratchet up their efforts to document progress and demonstrate results.
This is the third issue of The Evaluation Exchange dedicated to exploring the challenges and solutions associated with evaluating out-of-school time (OST) programs. It comes at a point in time when the need for credible OST research and evaluation information is at an all-time high, and when increased competition among nonprofits for scarce resources makes it more important than ever for OST programs to have the capacity to collect and use data for program accountability and improvement.
We focused this issue on what we see as necessary to continue to build the OST field’s capacity on these topics. This includes articles on what we know from existing research and evaluation about the results that are possible from OST programming, expert commentary on what the future OST research and evaluation agenda should look like, and information about hands-on research and evaluation tools and resources.
Included are articles about the implications for OST programs of the federal No Child Left Behind Act of 2001—particularly its emphasis on scientifically based research. Knowing that many of us do not have time to track and read every new report that comes out, we asked several authors to summarize key research and evaluation findings about what works in OST programming. Our Questions and Answers feature with Michelle Gambone, for example, offers exciting new findings from the longitudinal research she and colleague James Connell are conducting about the relationship between youth supports and opportunities and long-term developmental outcomes.
This issue also includes a special four-page feature on the implications of the recently released 21st Century Community Learning Centers (21st CCLC) first year evaluation report for future evaluation. Conducted by Mathematica Policy Research Inc., the evaluation examined the characteristics and outcomes of typical 21st CCLC programs. The first year findings, characterized as “disappointing” by the administration, were used to justify a proposed cut to future 21st CCLC programming.
This decision has stimulated renewed commitment to evaluation as well as efforts to examine this one evaluation in the larger context of other relevant evaluation and research. It also has catalyzed interest in developing a strong future research and evaluation agenda to support program development, improvement, and accountability. Therefore, in this issue HFRP provides information about a number of ongoing evaluations and additionally offers a special set of expert commentaries from researchers, evaluators, and practitioners about new directions for research and evaluation in order to reframe the “gotcha accountability” game into one of learning for continuous improvement and accountability.
In a world where the principles of scientific research are increasingly invoked to guide policy, the decision to use part one of an ongoing evaluation to cut funding is stimulating considerable scientific scrutiny and critique of the study’s methodology and findings. This scrutiny is crucial for scientific as well as policy advancement. Therefore, HFRP will publish an expanded set of commentaries later this spring that will be designed to examine issues in this particular evaluation and their implications for current policy as well as for future evaluation design, implementation, and use.
Responding to the many requests we receive for practical evaluation advice, we also included examples of evaluations in progress, information about HFRP’s expanding online database of OST evaluations, and a tool for OST logic modeling.
Because the space available in each Evaluation Exchange is limited, we encourage you to visit HFRP’s website (www.hfrp.org) for extra online information and resources. The online version of this issue includes an extensive New and Noteworthy section that we couldn’t fit in the print version.
Heather B. Weiss, Ed.D.
Founder & Director
Harvard Family Research Project