The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.
Volume XI, Number 2, Summer 2005
Issue Topic: Evaluation Methodology
From the Director's Desk
 |
| Heather Weiss |
A discipline's methodology, broadly defined, is its body of methods, approaches, rules, and techniques. Because evaluators come from diverse backgrounds, and because evaluation has become increasingly interdisciplinary, bringing points of view from communications, psychology, public health, and other arenas to bear, we enjoy a richness and variety in our methodology that is unmatched in many other disciplines.
While evaluation methodology is grounded firmly in traditional social science approaches, evaluation is more than the application of social science methods to study social problems. We make judgments and facilitate decisions that require us to regularly engage with programs and stakeholders and require us to integrate empirical evidence with standards and values to reach evaluative conclusions. As such, our methodology has evolved to embrace novel ideas and tactics that help us perform this complex work.
A fundamental benefit of having such a diverse methodology is that with every evaluation we have choices. The methods used for one evaluation can differ greatly from the next, depending on what we are trying to accomplish or answer and the theories and preferences we bring to the table. For example, if our priority is making sure our audiences use our work, we might choose a utilization-focused approach. If our priority is answering as unequivocally as possible “what works,” we may choose a randomized trial. If our priority is engaging stakeholders and building evaluation capacity, we may choose an empowerment or participatory approach.
Better yet, our choices are even broader because we can “mix and match” methodological designs (i.e., experimental, quasi-experimental, non-experimental) with different evaluation approaches (e.g., theory-based, goal free, constructivist), qualitative and quantitative methods, and useful tools and techniques (e.g., logic models, strategic planning, etc.). The possibilities are exciting, always growing, and help to keep our work fresh and engaging.
This is our third issue of The Evaluation Exchange devoted entirely to the theme of methodology, though with every issue we try to identify new methodological choices, the instructive ways in which people have applied or combined different methods, and emerging methodological trends. For example, lately we have seen “theories of change” gain almost buzzword status in our field, and have featured this concept in several recent issues, including this one.
Other topics explored here include the role that evaluation theory plays in our methodological choices, the proliferation of outcome models during the recent movement toward more accountability for social programs, how to give the connection between evaluation and learning more than just lip service, and developments in the never ending search to ensure our work avoids being forgotten on dusty bookshelves (or hard drives) and instead gets the attention we are so certain it deserves. We also highlight cluster evaluation and retrospective pretests as part of our methodological repertoire, and have several articles on the relationship between evaluation methodology, training, and minority populations.
Because evaluation methodology is constantly in flux, we will revisit this theme frequently in future issues. We will continue to keep our finger on the pulse of emerging trends and innovations, and as always, invite you to share your own observations about new developments or new methodological waters you've charted.
Heather B. Weiss, Ed.D.
Founder & Director
Harvard Family Research Project