The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.
Volume X, Number 3, Fall 2004
Issue Topic: Harnessing Technology for Evaluation
New & Noteworthy: Expanded Web Only Version
This section features an annotated list of papers, organizations, initiatives, and other resources related to the issue's theme.
The Association for the Study and Development of Community. (2004). A guide to evaluation primers. Princeton, NJ: The Robert Wood Johnson Foundation. This document is a guide to using handbooks and primers on program evaluation. www.rwjf.org/files/publications/RWJF_ResearchPrimer_0804.pdf
Barnard, R., Campbell, D. A., & Smith, S. (with Embuldeniya, D.). (2003). Citizen re:generation: Understanding active citizen engagement among Canada's information age generations. Toronto, Canada: D-Code. This study explores the interconnectedness of the ways young people engage with civil society and offers suggestions for those seeking to communicate with and involve young people in civic participation. www.d-code.com/pdfs/Citizencasestudies.pdf
Byers, C. (2002). Interactive assessment and course transformation using web-based tools. The Technology Source. This paper describes the roles web-based tools can play in active learning, interactive assessment, and closing the feedback loop in course assessment. technologysource.org/article/interactive_assessment_and_course_transformation_using_webbased_tools
Computerized Neighborhood Environment Tracking (ComNET) introduces handheld computers to community organizations to facilitate trained observer ratings. This methodology is being used in numerous cities across the United States. www.fcny.org/cmgp/comnet.htm
Education Week. (2004). Technology counts 2004: Global links: Lessons from the world. Bethesda, MD: Editorial Projects in Education. Education Week's seventh annual report on school technology presents an overview of technology in schools around the world. It also includes a review of national trends in the use of educational technology and state-by-state snapshots of the steps states have taken to use educational technology more effectively.
Ess, C., & Association of Internet Researchers. (2002). Ethical decision-making and Internet research: Recommendations from the AOIR ethics working committee. Association of Internet Researchers. This document offers a series of recommendations designed to support and inform those responsible for making decisions about the ethics of Internet research. aoir.org/reports/ethics.pdf
Haythornthwaite, C. (2004). Introduction: The Internet in everyday life. American Behavioral Scientist, 45(3), 363–382. This special issue of American Behavioral Scientist focuses on the impact of the Internet on access to resources, social interaction, and commitment to local community. The issue's introduction summarizes the results of studies and other data to examine and define the various ways the Internet affects everyday life.
Hoffman, J., & Clark, K. (2003). After school physical activity program evaluation report. Solana Beach, CA: Hoffman, Clark & Associates. This report summarizes the results of a comprehensive evaluation of the development of a website to provide after school staff with resources and training to increase the quality and quantity of physical activity in after school.
The Internet and Higher Education is a quarterly, peer-reviewed journal devoted to addressing developments related to learning, teaching, and administration on the Internet in postsecondary settings. www.elsevier.com/locate/iheduc
Johns, M. D., Chen, S. -L. S., & Hall, G. J. (Eds.). (2004). Online social research: Methods, issues, and ethics. New York: Peter Lang Publishing. This collection of essays by online researchers provides illustrations of how traditional research methods can be modified for effective online research. Authors also identify and discuss critical issues and dilemmas encountered in online research.
The Journal of Multidisciplinary Evaluation is a new, free online journal that aims to communicate current developments in evaluation to a diverse readership of evaluators, evaluation users, and would-be evaluators. evaluation.wmich.edu/jmde
Kosma, R., McGhee, R., Quellmalz, E., & Zalles, D. (2004). Closing the digital divide: Evaluation of the World Links program. International Journal of Educational Development, 24, 361–381. This article presents evaluation findings on the World Links program, an effort to provide schools in developing countries with networked computers and training to support the integration of technology into teaching.
Love, A. J. (2001). The future of evaluation: Catching rocks with cauldrons. American Journal of Evaluation, 22(3), 437–444. This article provides an overview of the profound ways in which developments in technology will affect the evaluation field.
Luckey, A. (2003). From improving IT infrastructures to achieving social good: Outcomes of nonprofit technology assistance providers (NTAPs). Seattle, WA: Blueprint Research and Design. This paper draws from the work Blueprint has done over the past 3 years with NTAPs and their funders to raise key issues regarding evaluation of IT assistance to nonprofits. www.blueprintrd.com/pubs.htm
McDonald, H., & Adam, S. (2003). A comparison of online and postal data collection methods in marketing research. Marketing Intelligence and Planning, 21(2), 85–96. This paper reports on a study which directly compares online and postal data collection methods using the same survey instrument on two samples drawn from the same population of football club subscribers.
MENTOR/National Mentoring Partnership's After-School Program Clearinghouse offers after school program coordinators a wide array of online resources to help them connect young people with existing mentoring opportunities, as well as develop new ones. www.mentoring.org/afterschool
Montgomery, K., Gottlieb-Robles, B., & Larson, G. O. (2004). Youth as e-citizens: Engaging the digital generation. Washington, DC: Center for Social Media. This report argues that an online youth civic culture has taken root on the Internet and is fostering participation in U.S. politics and community among those in “Generation Y”—those born after 1979. www.centerforsocialmedia.org/ecitizens/youthreport.pdf
The National Center for Education Statistics. (2004). State library agencies: Fiscal year 2002. Washington, DC: Author. This publication provides a statistical profile of state library agencies for state fiscal year 2002. The report includes information on staff, revenue, expenditures, services to libraries and systems, public service hours, service outlets, service and development transactions, collections, allied operations, and electronic services. nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2004304
National Center for Education Statistics. (2004). The status of public and private school library media centers in the United States: 1999–2000. Washington, DC: Author. This report summarizes findings from the public and private School Library Media Center Questionnaires, a component of the 1999–2000 Schools and Staffing Survey. Topics addressed include library media center characteristics, staff characteristics, library expenditures, collection holdings, and library policies. nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2004313
The National Science Foundation's Online Evaluation Resource Library was developed for professionals seeking to design, conduct, document, or review project evaluations. oerl.sri.com
Plant, R. (2004). Online communities. Technology in Society, 26(1). This article examines the concept and practice of online communities, first establishing their historical and technological roots, then developing a three-dimensional taxonomy through which they can be examined.
Rouse, C. E., & Kreuger, A. B. (2004). Putting computerized instruction to the test: A randomized evaluation of a “scientifically based” reading program. Economics of Education Review, 23, 323–338. This article presents results from an evaluation of Fast ForWord, an instructional computer program designed to improve language and reading skills. The randomized study assessed the program's impact on students having difficulty learning to read. www.sciencedirect.com
The School Information Partnership hosts a website, School results.org, which displays data for schools, districts, and states required to be publicly reported under the No Child Left Behind Act. The goal is to empower parents, educators, and policymakers to use the Act to make meaningful improvements in schools. www.schoolresults.org
Segone, M., Patel, M., Rouge, J. C., & Russon, C. (2003). Facilitating sustainable monitoring and evaluation capacity development. Brasilia: Brazilian Evaluation Network. This paper aims to examine the rapid, multinational process of strengthening monitoring and evaluation capacities that has been occurring over the last several years. A large part of the analysis is based on the experiences of the African Evaluation Association, and the National Associations and Networks in Africa. www.avaliabrasil.org.br/FirstpageRebramaEnglish.html
The State Educational Technology Directors Association (SETDA). (2003). 2003 SETDA National Leadership Institute toolkit: States helping states implement No Child Left Behind. This toolkit presents the outcomes of the second annual National Leadership Institute, a hands-on workshop with the goal of producing deliverables to aid state technology leaders in implementing the No Child Left Behind Act. www.setda.org/Toolkit2003/index.htm
Strack, R. W., Magill, C., & McDonagh, K. (2004). Engaging youth through photovoice. Health Promotion Practice, 5(1). This article describes the use of the photovoice method in an after school program. The photovoice method is a process that uses photos taken by people with little money, power, or status, to enhance community-needs assessments, empower participants, and induce change.
Strickland, O. L., Moloney, M. F., Dietrich, A. S., Myerburg, S., Cotsonis, G. A., & Johnson, R. V. (2003). Measurement issues related to data collection on the World Wide Web. Advances in Nursing Science, 26(4), 246–256. Using examples from the nursing field, this article identifies measurement issues related to Internet data collection that are relevant to qualitative and quantitative research approaches.
TechSoup, a one-stop resource for technology needs, provides free information, resources, and support. www.techsoup.org
University of Pittsburg, Office of Child Development. (2004). Internet-based solution to measuring outcomes is catching on. Developments, 18(1), 9–10. This article explores the use of Evaluation Station, an Internet-based tool developed for family service providers in Allegheny County, Pennsylvania, to measure the impact of services on the lives of troubled children and their families. www.education.pitt.edu/ocd/publications/dev2004-03.pdf
The U.S. Department of Education recently awarded a 5-year contract to Computer Sciences Corporation and its subcontractors to develop and operate a new database system for the Education Resources Information Center (ERIC). This new database will replace the existing one and will use the latest search and retrieval methods to cull education literature. Access will be granted to educators, researchers, and the general public. During the development and transition to the new ERIC, the current version will continue to be available at www.eric.ed.gov.
Using new technologies, the Carnegie Foundation's Knowledge Media Laboratory develops tools and resources to exchange information and produce innovations that can transform teaching and learning at many levels. www.carnegiefoundation.org/kml
Wentz, L. (2004). MindShare to read 20,000 media minds. Advertising Age. This article describes MindSet, a new initiative that tracks which communication channels consumers notice, where and when they notice them, and their frame of mind at the time of exposure. The channels covered by MindSet include all traditional media as well as direct marketing, retail, and transport advertising sites.
The William T. Grant Foundation's Consultation on Design of Group-Randomized Studies is a newly established consulting service created to assist in the design and analysis of group-randomized trials. sitemaker.umich.edu/group-based/consultation_service
Wolcott, J. (2004, Feb. 17). Wired schools help keep parents in the know. Christian Science Monitor. This article reports on the growing phenomenon of schools across the United States joining the online world with websites that post news about school events and allow parents and students to access teacher comments, test results, and grades. Retrieved October 16, 2004, from www.csmonitor.com/2004/0217/p11s01-legn.html
Wong, C. (2003.) How will the e-explosion affect how we do research? Phase I: The E-Del+I proof-of-concept exercise. Santa Monica, CA: RAND. This briefing documents a research project that used electronic tools to aid collaborative research. Researchers developed a taxonomy of electronic-aided research and conducted an exercise to show the feasibility of this approach. www.rand.org/publications/DB/DB399/index.html
New Resources From HFRPWe have recently published three new research briefs in our Issues and Opportunities in Out-of-School Time Evaluation series:
We have also published a new Snapshot in our Out-of-School Time Evaluation Snapshots series. Detangling Data Collection: Methods for Gathering Data describes the common data collection methods used by current OST programs to evaluate their implementation and outcomes. Both of these series of publications will be continued through 2006. To be notified when new briefs and Snapshots are available, sign up for our out-of-school time updates email. The Family Involvement Network of Educators (FINE) has added two new research digests and a new bibliography to the FINE website:
|
* Because of limitations on space, we were unable to include as many new and noteworthy resources as we would have liked in the print version of this issue. This online version includes all citations listed in the print version along with several others. If you like, you can sign up to be notified by email when the latest issue of The Evaluation Exchange is available online.
Erin Harris, Research Assistant, HFRP