Volume XI, Number 4, Winter 2005/2006

Issue Topic: Professional Development

Evaluations to Watch

Theory of Action in Practice

Claudia Weisburd and Tamara Sniad from Foundations, Inc. describe the use of a theory of change and a theory of action to help address questions about how to develop and evaluate professional development for after school staff.

For 10 years, Foundations, Inc. has been conducting professional development in the field of after school. Most recently, we have expanded our professional development activities from training providers on our written curricula to developing staff more broadly in the contexts of their own programs, including YMCA programs, parks and recreation programs, and many others. Last year, under a grant from the C. S. Mott Foundation, we offered professional development to close to 1,000 after school practitioners nationwide in a variety of full- and multiday formats, including a series of 3-day Afterschool Academies. All of our sessions present after school education as a distinct form of engaging with children and youth and offer core methodologies to facilitate learning and development.

As we describe below, we developed both a theory of change and a theory of action¹ to articulate our approach, define our intended impact, and design a comprehensive evaluation strategy to measure the impact of our professional development programs. In the long term, we aim to change practice and develop staff as educators in the field of after school.

Identifying Desired Outcomes of Professional Development
With help from the Bridgespan Group,² we modeled an overall theory of change (ToC) to represent our beliefs about how the professional development that we provide will lead to positive outcomes. Through the ToC development process, we identified our ultimate goal as improvement in learning and development for low-income children though their participation in high quality after school programs. The ToC highlighted that frontline staff skill is a core factor in high quality after school programs, along with other key factors, such as supervision, resources, and program infrastructure.

Identifying a Focused Set of Professional Development Activities and Staff Competencies
As part of our ToC, we developed a theory of action (ToA), or a plan for how our actions will affect the desired youth outcomes. These actions focus on our methods for building staff skills, as well as the program-level practices and characteristics that help staff to sustain what they have learned.

Our ToA pushed us to consider how to address staff who range widely in education and experience. We also faced the task of determining the fundamental competencies that define skilled after school educators. Unlike the field of education, in which instructors acquire an agreed-upon body of skills and knowledge via college curricula and supervised student teaching, an agreed-upon set of skills for after school providers is still evolving. Staff competencies that shape the content of our own workshops include competencies in linking to school content, building relation-ships, addressing the whole child, and using nonformal teaching methods.

Identifying Indicators of Change
Our ToA helped us develop indicators to measure whether our professional development has effects on practitioner and program levels—such as articulated and substantial learning objectives, positive learning environments and relationships, and resources devoted to staff development. We plan to evaluate degrees of change and measurement over time and relate that change to program infrastructure to capture other systemic factors.

Uncovering the Complexities of Evaluating Impact
Finally, the ToC and ToA process has highlighted the complexities of evaluating the impact of professional development in after school. First, direct professional development is one part of a complex system needed to improve staff skills and build program quality. Professional development sessions may be excellent, but without other parts of the system, effects may not be manifested or sustained in practice. Second, adult learning and change are ongoing processes, which may require a trial and error approach and time for ideas to percolate.

We are now conducting training in different “dosages” and with different groups. Our training content includes ongoing staff development to address the goal of sustained change. Evaluation data collected from close to 700 diverse participants suggests that this is the right track: Two thirds of our participants are responsible for training or supervising staff in their programs, and three quarters say they intend to use the materials we provide. A key component of our evolving outcomes assessment strategy is to collect follow-up data to identify whether intentions are translated into sustained action. Over the next 2 years, we hope to better understand what constitutes professional development that produces change, demonstrate the value of such professional development, and move the field toward improving staff capacities to provide high quality after school experiences for children and youth.

Click here for a larger version of the Foundations, Inc. Theory of Action.

¹ A theory of change identifies the process(es) through which a given type of social change is expected to occur. A theory of action maps out a specific pathway in that theory of change, or an organization's role with respect to achieving that change, based on an assessment of how it can add the most value to the change process.
² The Bridgespan Group is a nonprofit organization applying leading-edge management strategies, tools and talent to help other nonprofits and foundations achieve greater social impact. Their work with Foundations was generously supported by The Atlantic Philanthropies.

Claudia Weisburd, Ph.D.
Executive Director
Email: cweisburd@foundationsinc.org

Tamara Sniad, Ph.D.
Director
Training and Publications
Email: tsniad@foundationsinc.org

Center for Afterschool and Community Education
Foundations, Inc.
Moorestown West Corporate Center
2 Executive Drive, Suite 1
Moorestown, NJ 08057
Tel: 856-533-1600