Program Search Terms
Type:
Type:
Scope of Program:
Scope of Program:
Funding Level:
Funding Level:
- Select All -
Funding Sources:
Funding Sources:
Years in Operation:
Years in Operation:
- Select All -
Location of Program:
Location of Program:
Participants' Grade in School:
Participants' Grade in School:
Program Setting:
Program Setting:
Evaluation Search Terms
? Help
Experimental
Experimental designs all share one distinctive element: random assignment to treatment and control groups.
Experimental design is the strongest design choice when interested in establishing a cause-effect relationship. Experimental designs for evaluation prioritize the impartiality, accuracy, objectivity, and validity of the information generated. These studies look to make causal and generalizable statements about a population or impact on a population by a program or initiative.
Non-Experimental
Non-experimental studies use purposeful sampling techniques to get "information-rich" cases.
Non-experimental evaluation designs include: case studies, data collection and reporting for accountability, participatory approaches, theory-based/grounded-theory approaches, ethnographic approaches, and mixed method studies.
Quasi-Experimental
Most quasi-experimental designs are similar to experimental designs except that the subjects are not randomly assigned to either the experimental or the control group, or the researcher cannot control which group will get the treatment.
Like the experimental designs, quasi-experimental designs for evaluation prioritize the impartiality, accuracy, objectivity, and validity of the information generated. These studies look to make causal and generalizable statements about a population or impact on a population by a program or initiative.
Types of quasi-experimental designs include: comparison group pre-test/post-test design, time series and multiple time series designs, multiple time series designs, non-equivalent control group, and counterbalanced designs.
Design:
Design:
? Help
Document Review
This is a review and analysis of existing program records and other information collected by the program. The information analyzed in a document review was not gathered for the purpose of the evaluation.
Sources of information for document review include information on staff, budgets, rules and regulations, activities, schedules, attendance, meetings, recruitment, and annual reports.
Interviews/Focus Groups
Interviews and focus groups are conducted with evaluation and program/initiative stakeholders. These include, but are not limited to, staff, administrators, participants and their parents or families, funders, and community members.
Interviews and focus groups can be conducted in person or over the phone. Questions posed in interviews and focus groups are generally open-ended and responses are documented in full, through detailed note taking or transcription.
The purpose of interviews and of focus groups is to gather detailed descriptions, from a purposeful sample of stakeholders, of the program processes and the stakeholders' opinions of those processes.
Observation
Observation is an unobtrusive method for gathering information about how the program/initiative operates.
Observations can be highly structured, with protocols for recording specific behaviors at specific times, or unstructured, taking a more casual, "look-and-see" approach to understanding the day-to-day operation of the program.
Data from observations are used to supplement interviews and surveys in order to complete the description of the program/initiative and to verify information gathered through other methods.
Secondary Source/Data Review
These sources include data collected for other similar studies for comparison, large data sets such as the Longitudinal Study of American Youth, achievement data, court records, standardized test scores, and demographic data and trends.
Like the information analyzed in a document review, these data were not gathered with the purposes of the evaluation in mind; they are pre-existing data that inform the evaluation.
Surveys/Questionnaires
Surveys and questionnaires are also conducted with evaluation and program/initiative stakeholders. These are usually administered on paper, through the mail, in a highly structured interview process in which respondents are asked to choose answers from those predetermined on the survey, or more recently, through email and on the Web.
The purpose of surveys/questionnaires is to gather specific information.often regarding opinions or levels of satisfaction, in addition to demographic information.from a large, representative sample.
Tests/Assessments
These data sources include standardized test scores, psychometric tests, and other assessments of the program and its participants.
These data are collected with the purposes of the evaluation in mind. For example, the administration of achievement tests at certain intervals to gauge progress toward expected individual outcomes documented in the evaluation.
Methods:
Methods:
Evaluation Findings
? Help
Formative/Process Findings
Formative evaluations are conducted during program implementation in order to provide information that will strengthen or improve the program being studied.in this case, the out-of-school time program or initiative. Formative evaluation findings typically point to aspects of program implementation that can be improved for better results, like how services are provided, how staff are trained, or how leadership and staff decisions are made.
Formative/Process Findings:
Formative/Process Findings:
? Help
Sumative/Outcomes Findings
Summative evaluations are conducted either during or at the end of a program's implementation. They determine whether a program's intended outcomes have been achieved.in this case, the out-of-school time program or initiative's outcomes. Summative evaluation findings typically judge the overall effectiveness or "worth" of a program based on its success in achieving its outcomes, and are particularly important in determining whether a program should be continued. Summative outcomes can be short-term or longer term, depending on the purpose of the evaluation.
Summative/Outcomes:
Summative/Outcomes Findings:
Clear