The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.
Volume XII, Number 1 & 2, Fall 2006
Issue Topic: Building and Evaluating Out-of-School Time Connections
Evaluations to Watch
Efficacy Trials of Promising After School Programs: Supporting Experimental Studies Through a Research Consortium
Michael Vaden-Kiernan and Debra Hughes Jones from SEDL describe a U.S. Department of Education initiative to support rigorous research on the potential of after school programs to affect academic performance.
To date, rigorous attempts to gain empirical evidence concerning the impact of after school programs on student academic achievement outcomes have been mixed. An important empirical and policy-relevant question therefore remains: Can fully developed, well-implemented after school programs impact student academic achievement outcomes? Many stakeholders—from the policy, practice, and research communities—increasingly need this information.
To address this question, the U.S. Department of Education recently funded randomized controlled trials (RCT) that use rigorous methods to evaluate the benefits of promising after school interventions on student achievement in order to provide evidence that informs the national debate. The Southwest Educational Development Laboratory (SEDL), as part of the National Partnership for Quality Afterschool Learning, coordinated the proposal review process and is also leading a research consortium to support the studies over the 2-year funding period. Below, we describe the awardees' projects as well as the new research consortium. The following three studies received funding:
The Afterschool Training ToolkitCreated by The National Partnership for Quality Afterschool Learning (NPQAL) at SEDL, The Afterschool Training Toolkit is a staff development resource that shows after school practitioners how to create engaging activities that target specific academic content standards. Additional components on technology and homework help are being developed. In developing the tool kit, NPQAL researchers conducted and reviewed research, identified and visited high-quality after school sites across the U.S., and documented the connections between specific practices in each subject area and student achievement outcomes. Each subject section of the tool kit draws on all of these sources to feature sample lessons, interactive activities, and videos from after school programs that were identified by researchers as having a positive impact on student achievement. The Afterschool Training Toolkit is available online at www.sedl.org/afterschool/toolkits. |
- A Randomized Evaluation of the Adventure Island Afterschool Reading Program With English Language Learners, conducted by the Success for All Foundation. Adventure Island, an adaptation of the Success for All program, is currently being evaluated in a large-scale randomized experiment; however, that experiment includes few English language learners (ELLs). Because ELLs are a population of great interest, the new experiment will be conducted in 14 majority-Hispanic schools in Alabama, Texas, and Utah, with approximately 1260 children in grades 2–4.
- What Works in Afterschool Programs: The Impact of a Reading Intervention on Student Achievement in the Brockton Public Schools, undertaken by MPR Associates, Inc., in partnership with Scholastic Publications and Brockton (MA) Public Schools. This study is designed to compare Scholastic's READ 180 program, which has many characteristics that have been associated with positive academic outcomes, with Brockton's standard after school services. The study will include approximately 1,100 students in grades 4, 5, and 6, to provide methodologically rigorous information about READ 180 and to capture potential gains in reading skills and other outcomes.
- Afterschool Randomized Controlled Trials: The Voyager Passport Program in Kentucky 21st Century Community Learning Centers, designed by The Center for Evaluation and Education Policy at Indiana University, in collaboration with the Kentucky Department of Education. This study aims to compare the impact of previously established 21st Century Community Learning Centers (21st CCLC) programs with similar 21st CCLC programs that include the Voyager Passport Reading program (Passport). The study tests the hypothesis that economically disadvantaged youth in grades 2–5 who participate in a high-quality after school program with Passport achieve significantly greater learning gains over a 2-year period than do disadvantaged youth who participate in the same program without Passport.
SEDL will provide analytic and technical support to all three of these projects through the Afterschool Research Consortium (ARC). The ARC brings together SEDL staff, key staff from each research project, Technical Working Group (TWG) members, and representatives from the Department of Education on a regular basis to facilitate cross-fertilization of ideas and provide strategies to ensure full implementation of RCT designs in after school settings. The ARC focuses on substantive content issues in the after school field, technical and analytic issues in conducting rigorous RCTs, programmatic and research-to-practice issues, and policy issues. Project teams discuss ongoing implementation, data collection, and analytic challenges presented by the work; teams also explore opportunities to collaborate and disseminate the findings to the field.
Michael Vaden-Kiernan
Director of Research
Email: mvadenki@sedl.org
Debra Hughes Jones
Program Associate
Email: djones@sedl.org
SEDL
211 E. 7th St.
Austin, TX 78701
Tel: 1-800-476-6861
Website: www.sedl.org/afterschool