Volume X, Number 3, Fall 2004

Issue Topic: Harnessing Technology for Evaluation

Beyond Basic Training

Empowerment Evaluation's Technological Tools of the Trade

David Fetterman, from the Schools of Medicine and Education at Stanford University, describes how technological tools can be integrated into the practice of empowerment evaluation.

In less than a decade, empowerment evaluation has become a global phenomenon, reaching the four corners of the earth. Empowerment evaluation has also evolved: Definitions have been refined, principles have emerged, and practice has improved. Technology has been an integral part of the development of empowerment evaluation; the two have evolved side by side.1

Empowerment evaluation treats technology and its tools as more than simple mechanical devices or novelty gadgets. As described in this discussion, technological tools can greatly facilitate the conduct of empowerment evaluation at critical junctures in the evaluation process.

Empowerment Evaluation in Brief
Empowerment evaluation helps communities use self-evaluation and reflection to help themselves and improve their programs. In this type of evaluation, communities conduct their own evaluations while an external evaluator coaches or helps facilitate the process (Fetterman, 2001). The approach is guided by a set of 10 principles: (1) improvement, (2) community ownership, (3) inclusion, (4) democratic participation, (5) social justice, (6) community knowledge, (7) evidence-based strategies, (8) capacity building, (9) organizational learning, and (10) accountability (Fetterman & Wandersman, 2004).

The empowerment evaluation process involves three steps:2

1. Establishing the mission. The evaluator serves as a coach or critical friend, helping each community establish its mission or vision.
2. Taking stock. The evaluator helps the community take stock or assess where it is in its efforts. This serves as the community's baseline assessment.
3. Planning for the future. A plan is created for the future, with specific goals, strategies, and evidence. Traditional evaluation methods, such as interviews and surveys, are used cyclically to test whether strategies are working and to allow the community to make midcourse corrections. The community then conducts another assessment of its activities and compares findings to the initial baseline assessment to measure growth or change over time.

Using Technological Tools in Empowerment Evaluation
The principles and the three-step model help guide the use of technological tools in empowerment evaluation practice. The principle of inclusion, for example, might recommend the use of virtual conferences and videoconferences to include geographically remote groups during all three evaluation steps.3 During steps two and three, the capacity-building principle reminds the evaluator to provide community members with an opportunity to collect their own data, through online surveys, for example. The accountability principle guides community members to hold each other accountable for reaching specific standards or delivering specific results, products, or outcomes. Outcomes might be documented with QuickTime videos, digital pictures, and other technological tools with tremendous face validity.4

The discussion below describes in more detail some of these and other tools that can be used to facilitate empowerment evaluations. Examples are drawn from empowerment evaluations of the $15 million Hewlett-Packard Digital Village Program,5 the Hewlett Foundation's $5 million One East Palo Alto Project, and the Arkansas Department of Education's Rural Delta Academically Distressed School Districts Project (see homepage.mac.com/profdavidf for more details).6

• Digital movies. The entire empowerment evaluation process can be captured with digital pictures and digital videotape, and then used to produce movies that are placed on a community's website and burned as DVDs for large group presentations (Fetterman, 2002). The videos can be used as a form of documentation for funders concerning group or collaborative activity; provide transparency for the larger community by sharing what is being done and said; function as a reminder of group commitments; and serve as a marketing tool for additional funding.

• Videoconferencing. Videoconferencing can be used to share data, solve problems, exchange lessons learned, and build a bond between site personnel and the collaborating evaluation team. For example, one project's goal was to enable videoconferencing from an American Indian reservation to remote sites. The videoconference itself represented documentation of the group's accomplishments. Internet videoconferencing programs, such as CU-SeeMe, iVisit, and iChatAV (with an iSight camera/microphone), are free or inexpensive, with no long-distance charges or expensive equipment purchases (Fetterman, 1998).

• Web page, virtual conference, and file sharing. Empowerment evaluations are conducted locally. However, they typically require collaboration with external evaluators and other communities. Web pages, virtual conference spaces, and file sharing can facilitate communication across remote sites and constituencies. For example, in a hospital-based empowerment evaluation, health care providers were often pulled out of evaluation meetings for emergencies or were unable to attend important workshops because of previously scheduled activities. A web page and virtual conference space enabled nurses and physicians to keep up with transcribed mission statements, taking stock of ratings and dialogues, and the community's plans for the future.

  For the Hewlett-Packard Digital Village Program, the Stanford evaluation team maintained reports, digital photographs, and QuickTime videos in a virtual conference space on the Internet. The file-sharing capacity of this space enabled communities to draft, review, and approve reports (Fetterman, 1998). It provided around-the-clock accessibility of community-generated information and enhanced trust in the process.

• Online surveys. Online surveys are invaluable to empowerment evaluation. Communities have the capacity to survey large groups rapidly and inexpensively. The community helps determine the survey's purpose, specifies the topics, and recommends specific wording. The evaluator or coach helps refine the survey's questions and logical flow. Community members complete the survey online, saving mailing costs, data entry time and expense, and sorting time. Surveys are automatically entered into a database and sorted, providing the community with frequencies and bar graphs instantaneously (Fetterman, 1998).

• Picture and text messaging. Cell phones have the capacity to send pictures and text messages, and can be used to provide information about an ongoing problem or to document achievements. For example, one Arkansas Delta School District evaluation-team member sent a picture of students roaming the hallways during class time. This single photograph documented problems with student discipline and administrative follow-through.

  Digital photographs captured and transmitted over cell phones can also document program accomplishments. For example, a picture of a community-based organization distributing laptops to students and teachers was worth a thousand words. It documented one of the community strategies that emerged from the empowerment evaluation's plans for the future. Ease of access and use, and affordability, make camera phones an indispensable technological tool.

Although this discussion focused on the use of technological tools in empowerment evaluation, these tools can be applied to almost any evaluation. They are not a panacea and, like the dot-coms, require a healthy dose of skepticism and sensitivity. However, technology has the power to unleash tremendous untapped evaluation potential. That potential is restricted only by the limits of our imagination.

References
Fetterman, D. M. (1998). Webs of meaning: Computer and Internet resources for educational research and instruction. Educational Researcher, 27(3), 22–30. [Available at www.stanford.edu/~davidf/class/webs.htm.]

Fetterman, D. M. (2001). Foundations of empowerment evaluation. Thousand Oaks, CA: Sage.

Fetterman, D. M. (2002). Web surveys to digital movies: Technological tools of the trade. Educational Researcher, 31(6), 29–37.

Fetterman, D. M., & Wandersman, A. (Eds.). (2004). Empowerment evaluation principles in practice. New York: Guilford Publications.

1 See Fetterman (2001) for a chapter on the use of the Internet to disseminate empowerment evaluation worldwide.
2 See www.stanford.edu/~davidf/empowermentevaluation.html for more details and free guides.
3 Individuals without access to these tools are not excluded. Efforts are made to introduce technological tools when possible. The telephone, fax, and face-to-face communication are used when technology becomes a barrier.
4 Face validity is the extent to which videos, photos, etc., appear to provide valid documentation of a given outcome.
5 This project's aim is to help three communities leap across the digital divide: one African American urban community in the northeast, a Latino and African American low-income community in northern California, and 18 American Indian tribes in California. Projects included distributing laptops in schools, providing community centers with computers and Internet access, helping people grow small businesses, and developing web-based tools to enhance community access to local resources.
6 Special thanks are extended to Bess Stephens, Hewlett-Packard's vice president and global director for corporate philanthropy and education, and Dr. Charity Smith, assistant director of the Educational Accountability Section of the Arkansas Department of Education, for their support and guidance on these projects.

David Fetterman, Ph.D.

Director of Continuing Medical Education and Evaluation
School of Medicine
Tel: 650-269-5689

Director of Evaluation, Career Development, and Alumni Relations
School of Education
Tel: 650-723-1990

Stanford University
485 Lasuen Mall, 333
Stanford, CA 94305

Email: davidf@stanford.edu