Volume XV, Number 1, Spring 2010

Issue Topic: Scaling Impact

Promising Practices

Spreading Our WINGS: Using Performance Data to Prepare for Scale-Up

Many nonprofits collect data to measure their impact. But as Ginny Deerin, CEO of WINGS for kids, describes, they can also mine a treasure trove of performance data to improve their program models even before they undergo the scaling process.

The WINGS program began in 1996 and now serves three public elementary schools in impoverished Charleston, South Carolina, neighborhoods. WINGS is based on the premise that kids growing up in poverty often miss out on important life lessons—how to behave well, make good decisions, and build healthy relationships. WINGS instills those missing life lessons by weaving a comprehensive social and emotional education into a fresh and fun program that is delivered after school. Kids get the life lessons they need to succeed and thrive, and they get a safe place to call home after school.

To prepare for future scale-up, we mapped out a 5-year strategic plan. WINGS is now in the second year of an intensive research and development phase that requires building our organizational capacity, conducting a formative evaluation to assess our model, and conducting a summative evaluation to assess our impact.

At the heart of our evaluation efforts is a performance management system called Efforts to Outcomes (ETO). It tracks the progress of individual students toward the educational outcomes in our theory of change (TOC). Our TOC states that kids who receive a comprehensive social and emotional education for 15 hours a week within an after school program for 2 years or more will develop strong social and emotional skills and then use those skills in everyday life. It also states that they will demonstrate a strong attachment to school and consequently will graduate from high school, stay out of jail, and avoid teenage parenthood. In the long run, they will have a better chance for success and happiness in life.

The following are some of the short-term outcomes in our model that serve as stepping stones to long-term outcomes:

• Students demonstrate awareness of social and emotional skills. Indicator: 90% can recite the WINGS Creed, which reinforces the skills in kid-friendly language.
• Students understand the relevance of social and emotional teachings. Indicators: 80% can cite real-life examples of applying the Creed; 80% pass testing of the social and emotional skills learning objective for the week.
• Students demonstrate high attachment to school. Indicator: 95% receive average or higher grades for school attendance.
• Students use social and emotional skills in everyday life. Indicator: 85% receive positive social development grades on report cards.

ETO tracks each student’s progress toward the outcomes and allows WINGS staff to pinpoint difficulties in mastering the curriculum’s 30 learning objectives. Staff members known as WINGSLeaders carry notebooks around in which they jot down observations for each of the 10 to 12 students they supervise. They add these notations to computerized records daily.

Once we started using ETO, the data began showing us where we could make improvements in our model to better meet the needs of students. For example, WINGS staff learned that some students were not receiving positive social development grades from classroom teachers, even though our short-term outcome called for 85% to reach that target. In response, each WINGSLeader was required to develop a plan for the kids who were falling short and to share the plan with teachers in order to coordinate efforts to work with those kids.

The ETO data not only showed us deficits but also demonstrated whether the adjustments we made in response actually solved the problem. The graphic below gives an example of how ETO provides a snapshot of grades over a period of time, thus allowing us to check whether the next round of report cards indicates that our adjustments have made a difference.

By engaging in ongoing data review and assessment, WINGS has become far more responsive and accountable. Monitoring the data at frequent intervals means that WINGS does not wait until the end of the year, or even the end of a school term, to review how staff efforts are progressing. Most important, the continuous flow and analysis of information allow us to refine and develop our program model as we prepare to scale up.

Implementing a performance management system has yielded many benefits. Although we have been results-driven since our beginning, we have sharpened our focus through the process of clarifying what factors have been driving our educational outcomes. Detailed, easy-to-access records have allowed managers to monitor whether we have been implementing the program with fidelity to our model. Real-time feedback has provided powerful motivation for our staff in aligning efforts with outcomes. But by far the most significant gain is that this continuous flow of data provides us with valuable lessons in how to improve our program model to best meet student needs for maximum effectiveness.

Ginny Deerin
Founder and CEO
WINGS for kids
Email: ginny@wingsforkids.org