Volume X, Number 4, Winter 2004/2005

Issue Topic: Evaluating Family Involvement Programs

Evaluations to Watch

Learning From Parents Through Reflective Evaluation Practice

David Scheie, See Moua, and Pang Lee, from the Neighborhood Learning Community, describe the lessons they learned—about children participating in an affiliated program and about evaluation practice—by listening to parents’ spontaneous stories during a parent survey interview.

The Neighborhood Learning Community (NLC)¹ is a network of people and organizations working to strengthen the “culture of learning” in the West Side neighborhood of St. Paul, Minnesota. Based on the premise that everyone is a lifelong learner and teacher, the NLC promotes informal learning opportunities, sponsors leadership development, nurtures collaboration, and works for systems change, particularly within St. Paul public schools.

Evaluation is an integral part of the NLC. Consistent with the values of co-creation, collaboration, and reflective practice, the NLC uses a participatory approach. With this approach, information for learning sometimes bubbles up unexpectedly; this happened last spring when 10-minute parent surveys catalyzed hour-long conversations with parents about their children’s development and about the NLC.

Using a Parent Perceptions Survey—which contained multiple-choice questions for parents to rate their children’s development in seven areas—we interviewed Hmong parents whose children participate in an NLC-member organization known as the Jane Addams School for Democracy (JASD).² During the survey, parents rarely responded right away with one of the answer choices; instead, they told stories that taught us, as evaluators, about their children’s development and more.

From parents, we learned the following about the children at JASD:

• Each child is an individual and differs from his or her siblings in personal style and qualities, such as shyness, tidiness, or helpfulness to others.
• JASD children tend to be very self-expressive and committed to the community but have less awareness of the future.
• Children come to JASD to play and do homework but often don’t understand that JASD offers more than that.
• Children keep coming to JASD largely because of the relationships they build with each other and with adult staff.
• Many children fall short of their parents’ expectations at home with regards to responsibility, teamwork, and empathy, despite information from other sources that they behave well in these areas at JASD.

We also learned the following about parents and how to engage them:

• Parents trust JASD staff members, and this trust allows them to send their children to the program even when they don’t come themselves.
• Parents have a poor understanding of JASD, but the survey conversations increased their understanding of what the program provides children. These conversations confirmed that to reach out and engage parents effectively, we must meet with them face-to-face.
• Parents appreciated the space provided by survey conversations to talk positively about their children without being perceived as conceited or boastful. This appeared to reinforce their pride in their children, their confidence in themselves as parents, and their appreciation of JASD.

Using What We’ve Learned
These findings have helped strengthen our practice—our commitment to relationship building among children, parents, and staff; our emphasis on keeping adult staff involved long-term; and our encouragement of staff to initiate more one-to-one conversations with parents. We are also invested in learning more about why some dimensions of child development are further advanced than others through participation in JASD, why children behave better at JASD in some instances than at home, and how this information can be used to improve children’s learning.

We intend to foster more opportunities for parents to talk positively about their children and their hopes for them, as well as to engage them in reflecting with us about “what we did and why we did it,” so that parents and their children can begin to see JASD as not just a place to play and do homework, but a place tied to NLC’s core values of connection, co-creation, and public work. We also will continue involving parents in evaluating programming, as involvement helps parents better understand the program’s goals, begin to take ownership of the program, and assist program staff in co-creating the program.

Lessons for Evaluators
Our experience suggests the following lessons for evaluators:

• Be flexible and expect the unexpected. When parents “digress” into stories, we can learn from them in their own voice.
• Use both closed- and open-ended questions. We may revise our survey to include open-ended questions, because parents like that format. At the same time we’ll continue some closed items for easy quantification.
• Create teams to multiply insight. We really began to learn from parent comments when our two front-line surveyors did a reflective debrief with another evaluation-team member. We processed both parent comments and surveyors’ perceptions, cross-checking among ourselves.
• Use program staff and volunteers as data collectors to foster learning. Survey administrators’ familiarity with the program helped them create rapport that invited richer parental reflection.
• Harness linguistic and cultural affinity to gather richer information. Both of our parent surveyors were Hmong women who knew the language and culture of the immigrant parents they were surveying. We suspect this helped parents to open up.

¹ The NLC receives support from the Wallace Foundation, the Bush Foundation, and several other sources. For more information on the NLC, visit www.westsidelearning.org.
² See www.publicwork.org.

David M. Scheie
Evaluation & Collaborative Inquiry Consultant
Scheie & Associates
711 W. Lake Street, #411
Minneapolis, MN 55408
Tel: 612-825-9100
Email: dscheie@mtn.org

See E. Moua
Jane Addams School for Democracy
Center for Democracy & Citizenship
Hubert H. Humphrey Institute of Public Affairs
130 Humphrey Center
301 19th Avenue South
Minneapolis, MN 55455
Tel: 612-626-4699
Email: smoua@hhh.umn.edu

Pang K. Lee
Jane Addams School for Democracy
209 West Page Street
Saint Paul, MN 55107
Tel: 651-209-3519
Email: leex1104@umn.edu