Jump to:Page Content
You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.
The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.
September 17, 2013 Embracing the Use of Data for Continuous Program ImprovementHeidi Rosenberg
|
Evaluation for Continuous Improvement
This issue of the Evaluation Exchange examines the use of evaluation for continuous improvement, offers advice from well-known experts, and highlights innovative evaluation practices.
Data as a Flashlight
In this 4-minute video, Aimee Guidera, executive director of the Data Quality Campaign, discusses her theory that data “is a flashlight, not a hammer,” and can be used to shine a light on what is working well within schools in order to ensure continuous improvement.
FINE Newsletter, Volume V, Issue 3
Issue Topic: Creating a Culture of Continuous Improvement
Commentary
In this Commentary, Harvard Family Research Project’s Heidi Rosenberg discusses how continuous improvement processes can strengthen family engagement strategies, and outlines the mindset and key practices that organizations need to adopt in order to use data to understand and improve upon their work.
Schools, early childhood education programs, and youth- and family-serving organizations face growing pressure to demonstrate progress, with phrases such as “data-driven,” “outcomes-based,” and “results-oriented” peppering conversations about which programs or learning approaches should be adopted, funded, or otherwise prioritized. This demand for organizations to use data to demonstrate results often leads school, early childhood education, and other community leaders to view data collection and analysis as overwhelming, compliance-based tasks that generate anxiety rather than anticipation. Yet as Aimee Guidara, executive director of the Data Quality Campaign, put it in a related video:
We need to use data as a flashlight, not a hammer. In the past, education data has been used oftentimes as a blunt instrument … to make [educators] feel like they’ve failed … Data, when used well, can be used to shine a light on what’s working … This idea of data as a flashlight is absolutely critical to the idea of using data not only for compliance purposes, not only for accountability purposes, but for using information for continuous improvement.
Indeed, data can be used for much more than a summative judgment of whether a program “worked,” or didn’t. Using data for continuous improvement—collecting and using information on a program’s progress in an ongoing way, and involving multiple stakeholders in analyzing and applying the information gathered—allows educators and other practitioners to identify program needs and make immediate, real-time improvements to program services. Long used in the corporate sector, continuous improvement processes are now taking hold in the nonprofit arena, as organizations seek to understand how well they are serving their constituents and work to convince funders that their investments have been worthwhile.
What, then, does it mean to engage in a continuous improvement process? A recent post from Child Trends 5 notes that, in the field of early care and education:
Continuous improvement starts with program leaders who engage themselves and staff in reflecting on strengths and growth areas through self-assessments, feedback from colleagues and parents, and data collected about the quality of their program, classroom, or child care home. Professional development and technical assistance can be linked to growth areas, and programs as a whole can annually update goals, objectives and strategies for improving services.
Continuous improvement processes help answer the questions, “Who are we? What are we trying to do? What’s happening with our efforts? Are we making progress toward our goals?” More importantly, continuous improvement processes prompt organizations to think about how they will use data to improve the reach and quality of their services.1
HOW CAN CONTINUOUS IMPROVEMENT EFFORTS STRENGTHEN FAMILY ENGAGEMENT INITIATIVES?
Family engagement efforts are among the many initiatives to which continuous improvement processes can be applied, as schools and other organizations that serve children and families examine whether their strategies are making progress and if they are on track to achieving their desired outcomes.
Many schools, districts, early childhood education programs, and youth- and family-serving organizations have created family engagement initiatives to partner with families and work with them to identify students’ strengths and learning needs, and to connect families with relevant support services to enhance student achievement. But how do educators and other practitioners know whether their family engagement efforts are working—that is, whether they are reaching the families most in need of support, connecting families with the right services, or creating a school atmosphere that promotes the development of strong family partnerships? Equally important, how do educators know whether their family engagement efforts are helping students make progress toward their learning goals? Continuous improvement efforts give organizations the tools they need to strengthen their family engagement work as they go, which increases the chances of success and, subsequently, enhances support for their efforts among stakeholders.
One Head Start program referenced in this issue, for example, found that adopting a new computerized data-management system improved its ability to maintain and act on real-time information on family engagement, such as data on which parents have participated in different types of program services. This information allows staff members to quickly figure out which families they have not yet reached and design more intensive outreach strategies to connect with those families and help them obtain needed services. The data management system also allows different staff members who interact with a given family to understand what issues have already been addressed with the family so that staff can build on these efforts rather than just repeat them. The use of this system thus increases the efficiency of the program’s service delivery.
Using data for continuous improvement can also help educators and other practitioners understand why they might not be seeing the progress they hope to see. Understanding, for example, whether a program was implemented as planned can help practitioners pinpoint issues around service delivery that might affect students’ and families’ ability to make progress on outcomes. This type of “fidelity of implementation” data is critical when programs have multiple components that all need to work well in order for the initiative to succeed.
Gathering data on an ongoing basis also allows practitioners to troubleshoot problems as they arise, rather than waiting until the end of a program to realize that something did not go as planned. For example, in one instance a local United Way chapter that was implementing an initiative on family engagement in high school identified several significant barriers to family engagement and student success through the use of parent surveys. One important finding was that many parents relied on their teenage children to bring younger siblings to school, which often caused tardiness because elementary schools have a later start time than high schools. Many parents mistakenly believed that sibling care was a valid excuse for their older children’s tardiness. Learning this allowed the schools to speak directly with the families about this misunderstanding and to use the opportunity to help families understand how frequent tardinesses affected their children’s learning experiences in ways that could negatively impact academic success. This situation illustrates the fact that continuous improvement data not only allow practitioners to identify areas that need improvement, but also shed light on the kinds of improvement that would be most useful.
This process of ongoing data collection and use allows educators to make timely improvements to their family engagement efforts to ensure they are meeting the current needs of the families they serve. In this way, a school or community-based organization avoids expending time, effort, and funds on family engagement activities that might not be effective.
WHAT ARE THE FOUNDATIONS OF SUCCESSFUL CONTINUOUS IMPROVEMENT EFFORTS?
Continuous improvement is about making changes that enable an organization to realize its vision and achieve its goals. It is about taking steps to get better at leadership, staff development, relationships, curriculum, data systems, collaboration—the elements of an organization that work together to promote better outcomes for children and families. Using data for continuous improvement is most likely to succeed when certain attitudes and practices, as described below, are fostered and become integrated into the fabric of an organization.
At its heart, continuous improvement is about ensuring the quality and sustainability of an organization’s efforts. Beyond learning how to use the tools and carry out the technical aspects of continuous improvement processes, organizations need to establish a culture of inquiry. Doing so increases organizations’ capacity to use data to change for the better so that children and families benefit, staff members are engaged and empowered, and the organization comes closer to achieving its goals.
As you read this issue, we invite you to think about the following questions as you reflect on your own work:
1. How could a continuous improvement effort benefit my program?
2. How will this effort be structured so that people at different levels of an organization (e.g., direct service staff, program managers) or within a community access, share, and use data?
3. Are we making progress toward our family engagement outcomes? If not, why not?
4. What changes do we need to make to enhance our progress?
1Coffman, J., & Lopez, M.E. (2002). A conversation with Paul Light. The Evaluation Exchange, 8(2), 10–11. Available at http://hfrp.org/EE-EvaluationforContinuousImprovement
3 Mendel, C. B. (n.d.). Using data to inform program self-assessment and program improvement. Unpublished internal document, Early Childhood Learning & Knowledge Center.
This resource is part of the September 2013 FINE Newsletter. The FINE Newsletter shares the newest and best family involvement research and resources from Harvard Family Research Project and other field leaders. To access the archives of past issues, please visit www.hfrp.org/FINENewsletter.