Volume VIII, Number 2, Fall 2002

Issue Topic: Evaluation for Continuous Improvement

Promising Practices

Reflective Assessments: A Tool for Learning

Sharon Edwards and Ira Cutler of Cornerstone Consulting Group explain how organizations can use reflective assessments to assess their progress and consider the choices ahead.

In times like these of diminished resources, most funders expect the organizations they fund to evaluate their programs in some fashion. In some instances, particularly when organizations hope to establish a standard of practice or want to replicate their program, a formal, structured third-party evaluation, with appropriate controls, random selection, careful data collection, and measurement over time is the best method.

Many organizations, however, do not need and cannot afford an extensive third-party evaluation. Instead, their need is for a careful process that will provide them with the opportunity to stop and carefully reflect before moving on to the next stage. The best tool to use in this case is a reflective assessment.

Reflective assessments entail a short and inexpensive process through which organizations can assess their progress, indicate the lessons they have learned, document their accomplishments, and consider the choices ahead. Reflective assessments have been used in a number of ways by a variety of organizations:

• The Ford Foundation’s Community Youth Development Initiative and the Pew Charitable Trusts’ Neighborhood Preservation Initiative used reflective assessment to produce lessons learned reports, documenting the accomplishments of their initiatives for a broad audience.
• The Annie E. Casey Foundation’s Rebuilding Communities Initiative used reflective assessment to give a voice to participants’ view of the initiative to supplement the formal reports of external evaluators.
• The Los Angeles Children’s Policy Council (LACPC), approaching a 10-year milestone, used a reflective assessment process to build consensus and frame discussions about plans for the next 10 years.

Each reflective assessment is a separately designed project, but generally unfolds in five distinct phases. These phases are outlined below, using Cornerstone Consulting Group’s reflective assessment of LACPC as an illustrative example.

Phase One – Define the Engagement
In the first and in some ways most important phase, Cornerstone consultants meet with the leadership group to define the engagement: What is the purpose of the assessment? How will the findings of the assessment be used? Who is the audience? The LACPC sought a “lessons learned” report to inform the field, but, after a series of frank discussions, it decided on a different, more internally focused approach. Rather than focusing on exporting ideas, the LACPC decided it needed to take a deep organizational breath and think hard about options and goals for its next 10 years. Had the consultants not taken sufficient time at the front end to define the engagement, they might have produced a report to serve one purpose, but they would have missed the most important mark.

Phase Two – Establish an Information Base
Using readily available information, including annual and progress reports, mission statements, operating manuals, and formal evaluations, Cornerstone establishes an information base. In the case of the LACPC there was no lack of written information and, in fact, the challenge was figuring out what mattered most.

Phase Three – Capture Individual Perspectives
The heart of the reflective assessment process is a series of structured interviews with project leaders, stakeholders, and other constituency groups intended to capture individual perspectives. Cornerstone consultants spoke with 20–25 key informants in Los Angeles County for the LACPC reflective assessment. These interviews offer anonymity and time for personal reflection. Through dialogues with those closest to the effort, Cornerstone develops an analysis of achievements, challenges, the elements stakeholders consider most successful, and areas they see as problematic. The diversity of perspectives provides a voice for all groups involved in the effort. The interviews initiate and set the tone for a process of discovery and exploration and interviewees often say that the interview itself is an enjoyable learning experience.

Phase Four – Prepare an Assessment Report
Cornerstone prepares an assessment report providing an informed view of the initiative’s status and direction, and what has been learned during its course. This document is presented for review as a check on accuracy and tone before being finalized for broader distribution. Consultants ask if their picture is accurate, seek to clarify differences, and try to establish common ground. The report developed for the LACPC, The Tasks Ahead, raised questions that resonated with many CPC (Children’s Policy Council) members and created a “buzz” that focused attention on important issues.

Phase Five – Present the Report
On the basis of feedback and discussions, Cornerstone finalizes the report and often makes a presentation in a formal setting,reflect and react to the report. This meeting, and subsequent discussions, allow participants to focus on their core concerns: Are we on the right track? How do others view our effort? Are we encountering barriers that we didn’t anticipate? Should we make any changes in our approach? The assessment report does not sit on a shelf; rather it becomes a critical tool for charting the next steps.

While not the right tool for all efforts, at all times, reflective assessments fill an important niche in the evaluation landscape. A particularly important attribute has been the responsiveness of the process; a reflective assessment takes weeks, not years, to produce usable information.

Sharon Edwards
President
Cornerstone Consulting Group
One Greenway Plaza, Suite 550
Houston, TX 77046
Email: sedwards@cornerstone.to

Ira Cutler
Vice President
Cornerstone Consulting Group
181 Westchester Avenue
P.O. Box 791
Port Chester, NY 10573
Email: icutler@cornerstone.to