You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.

www.HFRP.org

The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.

Terms of Use ▼


FINE Newsletter, Volume V, Issue 3
Issue Topic: Creating a Culture of Continuous Improvement

 Commentary

In this Commentary, Harvard Family Research Project’s Heidi Rosenberg discusses how continuous improvement processes can strengthen family engagement strategies, and outlines the mindset and key practices that organizations need to adopt in order to use data to understand and improve upon their work.

Schools, early childhood education programs, and youth- and family-serving organizations face growing pressure to demonstrate progress, with phrases such as “data-driven,” “outcomes-based,” and “results-oriented” peppering conversations about which programs or learning approaches should be adopted, funded, or otherwise prioritized. This demand for organizations to use data to demonstrate results often leads school, early childhood education, and other community leaders to view data collection and analysis as overwhelming, compliance-based tasks that generate anxiety rather than anticipation. Yet as Aimee Guidara, executive director of the Data Quality Campaign, put it in a related video:

We need to use data as a flashlight, not a hammer. In the past, education data has been used oftentimes as a blunt instrument … to make [educators] feel like they’ve failed … Data, when used well, can be used to shine a light on what’s working … This idea of data as a flashlight is absolutely critical to the idea of using data not only for compliance purposes, not only for accountability purposes, but for using information for continuous improvement.

Indeed, data can be used for much more than a summative judgment of whether a program “worked,” or didn’t. Using data for continuous improvement—collecting and using information on a program’s progress in an ongoing way, and involving multiple stakeholders in analyzing and applying the information gathered—allows educators and other practitioners to identify program needs and make immediate, real-time improvements to program services. Long used in the corporate sector, continuous improvement processes are now taking hold in the nonprofit arena, as organizations seek to understand how well they are serving their constituents and work to convince funders that their investments have been worthwhile.

What, then, does it mean to engage in a continuous improvement process? A recent post from Child Trends 5 notes that, in the field of early care and education:

Continuous improvement starts with program leaders who engage themselves and staff in reflecting on strengths and growth areas through self-assessments, feedback from colleagues and parents, and data collected about the quality of their program, classroom, or child care home. Professional development and technical assistance can be linked to growth areas, and programs as a whole can annually update goals, objectives and strategies for improving services.

Continuous improvement processes help answer the questions, “Who are we? What are we trying to do? What’s happening with our efforts? Are we making progress toward our goals?” More importantly, continuous improvement processes prompt organizations to think about how they will use data to improve the reach and quality of their services.1

HOW CAN CONTINUOUS IMPROVEMENT EFFORTS STRENGTHEN FAMILY ENGAGEMENT INITIATIVES?

Family engagement efforts are among the many initiatives to which continuous improvement processes can be applied, as schools and other organizations that serve children and families examine whether their strategies are making progress and if they are on track to achieving their desired outcomes.   

Many schools, districts, early childhood education programs, and youth- and family-serving organizations have created family engagement initiatives to partner with families and work with them to identify students’ strengths and learning needs, and to connect families with relevant support services to enhance student achievement. But how do educators and other practitioners know whether their family engagement efforts are working—that is, whether they are reaching the families most in need of support, connecting families with the right services, or creating a school atmosphere that promotes the development of strong family partnerships? Equally important, how do educators know whether their family engagement efforts are helping students make progress toward their learning goals? Continuous improvement efforts give organizations the tools they need to strengthen their family engagement work as they go, which increases the chances of success and, subsequently, enhances support for their efforts among stakeholders.

One Head Start program referenced in this issue, for example, found that adopting a new computerized data-management system improved its ability to maintain and act on real-time information on family engagement, such as data on which parents have participated in different types of program services. This information allows staff members to quickly figure out which families they have not yet reached and design more intensive outreach strategies to connect with those families and help them obtain needed services. The data management system also allows different staff members who interact with a given family to understand what issues have already been addressed with the family so that staff can build on these efforts rather than just repeat them. The use of this system thus increases the efficiency of the program’s service delivery.  

Using data for continuous improvement can also help educators and other practitioners understand why they might not be seeing the progress they hope to see. Understanding, for example, whether a program was implemented as planned can help practitioners pinpoint issues around service delivery that might affect students’ and families’ ability to make progress on outcomes. This type of “fidelity of implementation” data is critical when programs have multiple components that all need to work well in order for the initiative to succeed.

Gathering data on an ongoing basis also allows practitioners to troubleshoot problems as they arise, rather than waiting until the end of a program to realize that something did not go as planned. For example, in one instance a local United Way chapter that was implementing an initiative on family engagement in high school identified several significant barriers to family engagement and student success through the use of parent surveys. One important finding was that many parents relied on their teenage children to bring younger siblings to school, which often caused tardiness because elementary schools have a later start time than high schools. Many parents mistakenly believed that sibling care was a valid excuse for their older children’s tardiness. Learning this allowed the schools to speak directly with the families about this misunderstanding and to use the opportunity to help families understand how frequent tardinesses affected their children’s learning experiences in ways that could negatively impact academic success. This situation illustrates the fact that continuous improvement data not only allow practitioners to identify areas that need improvement, but also shed light on the kinds of improvement that would be most useful.

This process of ongoing data collection and use allows educators to make timely improvements to their family engagement efforts to ensure they are meeting the current needs of the families they serve. In this way, a school or community-based organization avoids expending time, effort, and funds on family engagement activities that might not be effective.

WHAT ARE THE FOUNDATIONS OF SUCCESSFUL CONTINUOUS IMPROVEMENT EFFORTS?

Continuous improvement is about making changes that enable an organization to realize its vision and achieve its goals. It is about taking steps to get better at leadership, staff development, relationships, curriculum, data systems, collaboration—the elements of an organization that work together to promote better outcomes for children and families. Using data for continuous improvement is most likely to succeed when certain attitudes and practices, as described below, are fostered and become integrated into the fabric of an organization. 

  • A culture of inquiry. Organizations need to develop a culture that values inquiry, exploration, and self-examination.2 Doing so helps staff see data through the lens of discovery (e.g., What are the data telling me about my program’s efforts? How can I put the data to use?) rather than of compliance, and allows staff members to feel comfortable when discussing data rather than fearful that the data might make them look bad. Organizational leadership that commits to sharing data and providing support for staff to understand and use the information can support the growth of a culture of inquiry. Thus, over time, staff should be confident in using phrases such as, “The data tell us” and “The focus group findings suggest,” in place of phrases like, “I think” and “We feel” in discussions of their program and the children and families being served.3 

  • Staff engagement across all levels. All staff members of an organization need to become comfortable with, and competent in, using data to understand their program’s progress. Staff who work directly with children and families often gather data through child and family assessments, home visit observations, parent surveys, or focus groups. Those who supervise direct-service staff need to help them understand what the data say about children’s and families’ progress toward goals, and identify patterns among the data to troubleshoot delivery challenges or build on successful strategies. For example, in one early childhood program, a supervisor noticed that a child’s assessment scores suggested some developmental delays, yet there were no referrals to Early Intervention (EI) in the child’s file. Upon discussing this with the early education specialist who worked with the child, the supervisor learned that the child’s parents were resistant to the idea of EI because they were afraid their child’s involvement with EI would be stigmatizing. These data helped the supervisor work with the early education specialist to devise a specific outreach strategy to help the parents understand the benefits of EI and connect their child with needed services. Program leaders can also aggregate data to assess a program’s collective progress with the children and families it serves, including progress across multiple sites, to identify patterns of efficiency and effectiveness. Additionally, program leaders can also use these data to develop targeted professional-development strategies that address specific needs or are tailored to specific staff groups. 
  • Mechanisms for organizing, sharing, and reflecting on data. Programs need to have systems in place that allow staff to enter, manage, aggregate, and report on data. Such systems allow for the centralized organization of information pertaining to a given child or family so that staff can obtain a “big-picture” sense of what services a family has received, what issues have been addressed, and what progress has been made thus far. Program leaders also need to establish processes that allow for dedicated time for data review and discussion. These can include:
    • Reflective supervision in which supervisors discuss the data that a staff member has collected and help the staff member understand how to put the data to use to improve services;
    • Regular staff meetings in which staff review children’s or families’ progress and discuss what the data suggest about needed support services or other targeted interventions; and
    • Annual full-staff retreats that are dedicated to reviewing data about the organization’s progress across programs and sites, and discussing how the data can inform strategic planning for the following year. 
  • Realistic and incremental goals. Programs need to think about different levels of data collection and analysis so that data management isn’t seen as simply an overwhelming and time-consuming effort. While the long-term, “big-picture” goals may be the focus of a funder’s, or the public’s, assessment of a program’s effectiveness, organizations will benefit from interim outcome data that show whether they are on track to achieving these long-term goals. For example, when United Way Worldwide launched an initiative to promote family engagement during the high school years, the overall goal was to improve high school graduation rates. Many project sites chose to focus their interventions on ninth graders and their families. The programs developed interim indicators of progress toward high school graduation, such as attendance, grades, and credit accumulation. Data on these indicators allowed sites to track progress along the way rather than waiting for actual graduation data, and allowed them to identify what was working well and what needed to be adjusted in their strategies. 

  • Inclusion of key stakeholders in data sharing and review. Organizations need to help key stakeholders understand the purpose and context of findings from data analysis. Organizations also need to create opportunities to reflect upon the data with stakeholders and develop a shared understanding of how best to act on the findings. Having regular opportunities for discussion and reflection with stakeholders helps organizations build a collective understanding of and support for a program’s work. For example, in the Federal Way school district in Washington State, the district’s Family Partnership Advocate (FPA) and the superintendent hold quarterly meetings with school staff, parents, and community members where they discuss a broad range of topics, including the progress of the district’s family engagement activities. Discussions on these activities include steps that the district can take to make improvements in its programs and ways that staff can further strengthen their family engagement work. After each meeting, the FPA creates a summary that is shared with school personnel, families, and the community. Doing so helps to build a broad base of community support for the district’s family engagement strategies and enhances buy-in from multiple stakeholders.

At its heart, continuous improvement is about ensuring the quality and sustainability of an organization’s efforts. Beyond learning how to use the tools and carry out the technical aspects of continuous improvement processes, organizations need to establish a culture of inquiry. Doing so increases organizations’ capacity to use data to change for the better so that children and families benefit, staff members are engaged and empowered, and the organization comes closer to achieving its goals.

As you read this issue, we invite you to think about the following questions as you reflect on your own work:

1. How could a continuous improvement effort benefit my program?

2. How will this effort be structured so that people at different levels of an organization (e.g., direct service staff, program managers) or within a community access, share, and use data?

3. Are we making progress toward our family engagement outcomes? If not, why not?

4. What changes do we need to make to enhance our progress?



1Coffman, J., & Lopez, M.E. (2002). A conversation with Paul Light. The Evaluation Exchange, 8(2), 10–11. Available at http://hfrp.org/EE-EvaluationforContinuousImprovement

2 Dykman, A. (2002). Mindset matters. The Evaluation Exchange, 8(2), 2–3. Available at http://hfrp.org/EE-EvaluationforContinuousImprovement

3 Mendel, C. B. (n.d.). Using data to inform program self-assessment and program improvement. Unpublished internal document, Early Childhood Learning & Knowledge Center.


Click here to access other articles and resources in this FINE Newsletter, "Creating a Culture of Continuous Improvement."

 


This resource is part of the September 2013 FINE Newsletter. The FINE Newsletter shares the newest and best family involvement research and resources from Harvard Family Research Project and other field leaders. To access the archives of past issues, please visit www.hfrp.org/FINENewsletter

© 2016 Presidents and Fellows of Harvard College
Published by Harvard Family Research Project