The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.
Volume IX, Number 1, Spring 2003
Issue Topic: Evaluating Out-of-School Time
Evaluations to Watch
Improving Bradenton’s After School Programs Through Utilization-Focused Evaluation
The John S. and James L. Knight Foundation’s Community Partners Program is working to improve the quality of life in 26 U.S. communities where the Knight brothers owned newspapers. Katherine Loflin of the Knight Foundation describes the utilization-focused evaluation efforts in one of Knight’s communities.
Building on existing local capacity, the Knight Foundation and a local advisory group selected to invest $1.75 million in local partnerships in Bradenton, Florida over the next five years to improve positive youth development for students at two middle schools. The investment is in after school programs serving each of the middle schools. Goals of these programs are to: (1) increase youth school engagement, (2) decrease youth negative and violent behaviors, and (3) increase youth civic engagement.
After consultation with local stakeholders, Knight contracted Philliber Research Associates (PRA)¹ to help plan and evaluate each after school program. Providers in the after school programs formed a working collaborative and, with Knight Foundation assistance, formulated a utilization-focused evaluation plan² that will use data collected from young people at both the school and program levels to inform program practice over the next five years.
A Utilization-Focused Evaluation Plan
The after school collaboratives, Knight, and PRA developed a school-wide survey that will be administered to all middle school students at the beginning and end of the school year for the next five years. The survey collects information on: demographics/family status, school engagement (asking questions such as: How important is it to you to do well in school? Do you have a quiet place to do homework?), school engagement indicators (such as frequency of truancy, not completing homework), risky behaviors (such as lack of supervision, substance abuse), self-concept, current after school involvement, and other extracurricular involvement (including volunteering, athletics, school clubs). Additionally, at the end of each school year, PRA will collect academic performance data for each child attending the after school program. PRA will join these two data sets with student program attendance data to analyze any changes and the potential impact of program attendance on the changes observed.
The school-wide survey, which provides baseline data on all potential program participants, was completed in December 2002. Baseline data at the school level are critical for two reasons. First, the data provide a starting point for after school program providers to fine-tune their services by showing the needs of program participants. Second, the after school collaboratives will use these data for program improvement and, in consultation with Knight, to develop future grant proposals that reflect program modifications.
Next Steps
PRA will continue to work with Knight’s grant partners in Bradenton throughout the school year to monitor and evaluate the after school programs and to use the year-end school data to inform additional program modifications. Knight Foundation and its grant partners in Bradenton, with the help of Philliber Research Associates, will continue to build capacity for utilization-focused evaluation to improve after school program quality in Bradenton.
Katherine Loflin
Content Program Officer in Civic Engagement
John S. and James L. Knight Foundation
Wachovia Financial Center, Ste. 3300
200 South Biscayne Blvd.
Miami, FL 33131
Tel: 305-908-2600
Email: loflin@knightfdn.org
¹ For more information, see www.philliberresearch.com.
² Defined as evaluation done for and in collaboration with the primary users (in this case, after school program providers) for specific and intended uses (in this case, to monitor and make program choices for the after school programs).