The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.
Volume XII, Number 1 & 2, Fall 2006
Issue Topic: Building and Evaluating Out-of-School Time Connections
Evaluations to Watch
An Emerging Assessment of College Preparation, Media, and Arts Programs
Karen Walking Eagle, Sebastian Castrechini, and Monica Mielke from Policy Studies Associates preview a new assessment of programs that connect youth with multiple out-of-school supports to promote future success.
Through its College Prep and Youth Media and Arts Fund grant programs, Time Warner Inc. supports 21 out-of-school time programs in New York City. These programs seek to strengthen the academic, creative, artistic, and life skills of underserved public-school youth and to prepare them to succeed in high school, college, and careers. The programs connect youth with academic opportunities and social exposure more frequently available to their more affluent peers. Time Warner Inc. has retained Policy Studies Associates, Inc. to assess the extent to which these programs have developed and implemented processes and features that promote positive outcomes among participating youth.
Seven of these 21 programs are funded under Time Warner Inc.'s College Prep grant program, which seeks to increase academic achievement and college enrollment among targeted students. Youth typically enroll in these programs while in middle or high school and attend academic enrichment classes after school and, in some cases, on weekends and during the summer. All seven programs provide test preparation and instruction in key learning skills and core subjects such as math and language arts; many also provide counseling and mentoring about the college application process, as well as leadership training, internships, and travel and volunteer opportunities.
The Achievement Gap InitiativeLed by scholars Ronald Ferguson, Richard Murnane, and Charles Ogletree, the Achievement Gap Initiative (AGI) at Harvard University focuses on bringing together academic research and public attention to eliminate gaps in achievement between disadvantaged and minority youth and their more advantaged peers. Research suggests that a variety of factors, including early childhood education and parental involvement, explain the persistence of achievement gaps and that building complementary learning supports can help raise achievement. With major funding from Time Warner Inc. and additional support from the Spencer Foundation, Murphy Foundation, and Harvard Graduate School of Education, AGI promotes forums for sharing research and for conducting outreach and public education. AGI recently created a website that offers a variety of resources, including multimedia coverage of past events. Additionally, the website provides an overview of the broad base of research about achievement gaps and offers suggestions for how institutions and individuals involved with youth can narrow gaps in achievement. Many of these potential solutions entail purposeful linkages to support youth. For more information about the Achievement Gap Initiative, visit agi.harvard.edu/mission.php. |
Policy Studies Associates is conducting an assessment of all 21 Time Warner Inc.-funded programs. The assessment will report on grantees' estimates of program results, obtain youth input on their program experiences and effects, and will determine the extent to which the programs funded under these initiatives possess certain features that have been shown by existing research to promote learning and other types of positive youth development. These features include practices that promote positive relationships, rich content-based program activities, and learning- and mastery-oriented content-delivery strategies, including youth choice and leadership opportunities.
The assessment will also examine key structural and institutional features that advance the desired process, as well as content features, such as recruitment and retention strategies for both participants and staff, the careful alignment of program goals and activities, and connections to family and community. For example, based on interview data collected from youth participants and program staff, we will describe how professionals in the community have shared their knowledge with participants and the reported impact of this on participants' technical expertise, content knowledge, personal aspirations, and career goals.
We have completed data collection for the assessment. The data collected include individual project reports; interviews with selected project leaders, staff, and youth participants at eight sites; a survey of all 21 project directors; and on-site observations of participant activities at the sites. Because research shows that staff play a critical role in promoting quality programs, we asked project directors about staffing issues, including recruitment practices, training opportunities, and supervision and assessment policies. We also asked program staff about the same factors, as well as about conditions that impede or facilitate their work and the strategies they use to overcome challenges. Conversations with staff have enabled us to take a closer look at the importance of partnerships and at the roles played by mentors, parents, families, and the community.
In both the surveys and site visits, we collected data on the reported academic, artistic, creative, social, and psychological outcomes among participants. For example, during interviews, we asked participants how they benefited from participating in the program and encouraged them to provide specific examples. Youth were also asked to identify those program elements that were most effective in helping them to improve academically, artistically, or socially. In surveys, program directors were also asked to describe the areas in which significant numbers of participants demonstrated desired outcomes. Our report will draw on our site visit observations and on staff and participant reports to describe those program features and practices associated with these outcomes—in particular, the strengthening of academic and life skills that prepare youth for future success.
Karen Panton Walking Eagle
Senior Research Associate
Tel: 202-939-5339
Email: walking-eagle@policystudies.com
Sebastian Castrechini
Research Analyst
Tel: 202-939-9311
Email: scastrechini@policystudies.com
Monica B. Mielke
Research Analyst
Tel: 202-939-9320
Email: mmielke@policystudies.com
Policy Studies Associates, Inc.
1718 Connecticut Avenue NW, Suite 400
Washington, DC 20009