The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.
Volume XV, Number 1, Spring 2010
Issue Topic: Scaling Impact
Promising Practices
The RALLY Program: Scaling an Inclusive Approach to Intervention and Prevention
Helen Janc Malone of Harvard Family Research Project describes an after school program’s strategy for scaling its services and the role of evaluation in the scaling process.
The Responsive Advocacy for Life and Learning in Youth (RALLY) after school program takes an unconventional approach to scaling. Accord-ing to RALLY’s director, Gil Noam, the after school field often assumes that scaling means replicating programs. What can get lost in that ap-proach, however, is the importance of local context, charismatic leaders, motivated staff, a mission targeted to local needs, family involvement, and youth engagement in making a program a success. Noam argues that rather than scaling up a cookie-cutter model, after school programs should invest in scaling up a model that can be customized to address the needs of local youth. As Noam explains, “People do not want another Starbucks—they want the old mom-and-pop shops.”
Core Principles and Customization
RALLY, started at a middle school in Jamaica Plain, Massachusetts, has scaled its approach in demonstration sites at three more middle schools, in Honolulu, Hawaii; Syracuse, New York; and Tacoma, Washington. All of the sites apply RALLY’s core principles—they offer school-based integrated services to encourage learning, promote social-emotional development, foster trusting and caring youth–adult relationships, and build youth resilience and interpersonal skills. Noam recalls communicating the RALLY approach to the individual sites: “These are the practices that we have found to be helpful and useful at RALLY. If you apply these practices to the staff you have, then you can be successful too.” At the same time, each site offers its own variety of activities customized to fit each school’s needs, including classroom support, tutoring, mentoring, enrich-ment opportunities, referrals, and professional crisis management.
The success of RALLY’s approach to scaling relies in part on the selection of the sites themselves. The after school groups and school dis-tricts, with which RALLY partners, must have in place the internal capacity to align the RALLY program’s approaches with their organizational missions. RALLY thus requires each site to have, at a minimum, a full-time on-site coordinator, a mental health professional, trained social work-ers and counselors, a strong program–school partnership, and relationships with community support services. This requisite capacity, coupled with content and organizational flexibility, allows each site to create its own strategic and business plan that considers contextual factors, adheres to the expectations of the local school district, and links to locally desired youth outcomes. As Noam explains, RALLY’s approach to scaling has allowed each of the sites to operate as a cohort, exchanging ideas that work for that community.
Evaluation Strategy
The RALLY scaling method has important implications for its evaluation strategy. Flexibility in local implementation allows for a customized evaluation at each site. According to Noam, many after school programs that operate in multiple sites engage in evaluations that focus mainly on accountability (i.e., to satisfy a funder). These evaluations rely heavily on standardized measures that do not adequately assess individual pro-grams’ desired outcomes; their reporting time often lags months behind when the data were collected, diminishing the ability of site staff to make continuous, informed decisions about activities and services. A more beneficial approach, Noam suggests, is to focus on data-driven programming where the evaluation is directly connected to youth experiences at the program sites. Such evaluation strategies create a feedback loop around youth experiences, staff expectations, and program goals. Feedback from frequent participants in programs helps guide management decisions, aids in the construction of activities that engage youth in learning, and improves program services and supports.
Scaling up programs across multiple sites is a popular practice in the after school field, explains Noam, but “letting all flowers bloom” requires investing in a strong organizational infrastructure, developing strategic plans on how to approach scale, and engaging in rigorous evaluation that offers data on the quality of implementation across sites. For programs with the capacity to scale up to multiple sites, employing a flexible pro-gram model that encourages local adaptation in services, activities, and evaluation strategies can create a sense of ownership at the sites. It can also create a cohort model of learning that encourages individual site improvement and growth and, potentially, an increased quality of the pro-gram as a whole.
Helen Janc Malone
Graduate Research Assistant, Harvard Family Research Project
Email: hjanc@mac.com
Related ResourcesProgram in Education, Afterschool & Resiliency (PEAR). Dedicated to a whole-child approach, PEAR integrates research, theory, and practice across youth development, school reform, and mental health fields. Gil Noam, PEAR’s director, is also the editor-in-chief of New Directions for Youth Development, a quarterly journal dedicated to contemporary issues in the field of youth development. To learn more, visit: www.pearweb.org. More information about the RALLY program can be found at: www.pearweb.org/rally.Assessment Tools in Informal Science (ATIS). PEAR’s new database, ATIS, offers assessment tools for after school programs. Users can search for tools by age, domain, type, or custom criteria. Each tool provides users with a short description, psychometric properties, and an author contact. Avail-able instruments can quickly be applied in after school settings to start program- and youth-level data collection. To access the database, visit: www.pearweb.org/atis/. |