The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.
Volume VIII, Number 2, Fall 2002
Issue Topic: Evaluation for Continuous Improvement
Ask the Expert
What Is a Learning Approach to Evaluation?
Rosalie T. Torres, Ph.D. is Director of Research, Evaluation, and Organizational Learning at the Developmental Studies Center in Oakland, California. Her 24-year career in evaluation has focused on researching, teaching, writing about, and practicing a learning approach to evaluation.
A learning approach to evaluation is contextually-sensitive and ongoing, and supports dialogue, reflection, and decision making based on evaluation findings (Torres & Preskill, 2001). Its primary purpose is to support learning that can ultimately lead to effective decision making and improvement in department, programmatic, and organization-wide practices. It is based on the premise that to achieve learning and improvement, evaluation must:
- Consider (but not be completely governed by) the context in which it is being conducted (i.e., stakeholders’ information needs, political realities, organizational culture).
- Be conducted routinely.
- Provide opportunities for stakeholder participation—in particular, opportunities to ask questions about, discuss, and reflect on the meaning of evaluation findings.
A learning approach can be taken with any kind of evaluation. The goal is to conduct the most valid, relevant, and credible evaluation possible—given the information needs and the existing political, logistical, and resource constraints—and to actively facilitate learning from the evaluation and taking action. This means establishing a balance between accountability and learning roles for evaluation.
Learning occurs best among individuals who regard the information they are reviewing (i.e., evaluation findings) as credible and relevant to their needs. Involving stakeholders in designing and conducting an evaluation helps assure their ownership of, and interest in, its findings.¹ Learning also occurs best among individuals who have an opportunity to ask questions about evaluation methods, consider other sources of information about the topic in question (including their own direct experiences), and at the same time hear others’ perspectives.
One way to take a learning approach with any evaluation, whether you have been involved in conducting it or not, is to enable (or plan for) deep consideration of its findings and the planning of next steps. Often evaluators and their clients think the major work of an evaluation is complete when the final evaluation report has been written. To the contrary, some of the most productive work of an evaluation occurs when stakeholders convene to review and discuss findings and implications. The meeting should be conducted with the key principles of adult learning and staff development in mind: have participants engage with the learning material; see, hear, and do something with its content; and integrate new knowledge with what they already know (Torres, Preskill, & Piontek, 1996).
Consider these guidelines for working sessions of several hours or even over the course of a daylong retreat to review, discuss, and interpret evaluation findings:
- Begin with a presentation of findings via posters, overheads, a PowerPoint presentation, or other form of visual display so that participants receive and process the findings together rather than individually.
- Give sufficient opportunity for participants to ask questions about the evaluation procedures and data analysis.
- Provide an interactive experience for participants to engage with the findings. For example, have participants work in small groups to develop conclusions based on the findings. (The groups might each take a different section of the findings.)
- Have each group present their conclusions, and give the evaluator an opportunity to comment on or raise questions about conclusions that may have been missed or overstated.
- Post findings and conclusions around the room and have groups of participants visit each poster, recording their thoughts about implications and actions to consider.
- Debrief in a large group discussion on the implications and actions.
- Conclude with a more detailed action-planning activity or definite next steps which will lead to action planning.
Rosalie T. Torres, Ph.D.
Director of Research, Evaluation, and Organizational Learning
Developmental Studies Center
2000 Embarcadero, Suite 305
Oakland, CA 94606
Tel: 510-533-0213 x235
Email: rtorres@devstu.org
¹ See Patton, 1997 and Preskill & Torres, 1999 for more about stakeholder involvement in all phases of evaluation.
References
Patton, M. Q. (1997). Utilization-focused evaluation: The
new century text (3rd ed.). Thousand Oaks, CA: Sage.
Preskill, H., & Torres, R. T. (1999). Evaluative inquiry for learning in organizations. Thousand Oaks, CA: Sage.
Torres, R. T., & Preskill, H. (2001). Evaluation and organizational learning: Past, present, and future. American Journal of Evaluation, 22(3), 387-395.
Torres, R. T., Preskill, H., & Piontek, M. E. (1996). Evaluation strategies for communicating and reporting: Enhancing learning in organizations. Thousand Oaks, CA: Sage.