Learning from what works
What is evaluation?
Evaluation is a systematic assessment of the design, implementation, or outcomes of a program. Evaluations can be used to:
- develop new programs or adapt existing ones
- learn from what works (as well as what does not for who and how)
- make decisions about future programming
- share learnings so others can build on them
Types of evaluation
Formative
When to use
- When developing a new program
- When modifying a program for a new setting or population
Research questions
- Is the program being delivered as intended to the targeted recipients?
- Does anything about the program need to be changed in order to meet its overall goals?
Example
- This formative evaluation included a needs assessment and an implementation analysis to improve the effectiveness of Patient-Centered Medical Home Interventions.
Process
When to use
- From the very beginning of program implementation
- Through operation
Research questions
- Have any unanticipated administrative issues emerged?
- Are program resources being used efficiently and effectively?
- Is anything taking longer than anticipated?
Example
- The Center for Applied Research at the University of Connecticut for the State of Connecticut’s Office of Policy and Management conducted a process evaluation that included collecting attendance and survey data to improve a Neighborhood Youth Center Program.
Outcome
When to use
Research questions
- Is the program having a positive or negative impact on the identified recipients?
- Are desired program outcomes obtained?
- What, if any, unintended side effects did the program produce?
- Do outcomes differ across program approaches, components, or groups of individuals?
Example
- The Center for Employment Opportunities (CEO) partnered with independent research firm MDRC to conduct a process and outcome study of their Returning Citizens Stimulus (RCS) which found that cash assistance combined with reentry services has strong potential to improve economic mobility outcomes.
Impact
When to use
- During program operation
- At the end of a program
Research questions
- Will the program continue to be effective beyond the life of the project?
- Did the program cause the desired impact?
- Is one approach more effective than another in obtaining the desired outcomes?
- Were groups beyond the intended recipients impacted by this program?
Example
- The National Institute of Justice conducted and evaluation to assess the impact of adult drug courts located in various sites across the United States.
Rigorous/Randomized Control
When to use
- At the end of a program
- When you want to determine causal effect and you have both an intervention group and a control group
Research questions
- Did my program have the intended impact?
- What was the extent of the impact?
Example
- The National Cancer Institute conducted a randomized control trial to understand the effectiveness of Students for Nutrition and eXercise (SNaX), a school-wide obesity prevention program for middle school students that promotes healthy eating and physical activity.
Have questions? Contact us