Frequently asked questions

I want to implement a new program – how I can find one that has already been evaluated and shown to work?

Programs that have demonstrated positive results in independent, rigorous evaluations are often referred to as "evidence-based” programs. Results for America’s Economic Mobility Catalog was created specifically to help local government leaders identify evidence-based approaches to their economic mobility-related challenges.

It draws from the following widely used government and philanthropic “clearing house” resources:

• The Pew-MacArthur Results First Initiative
• The U.S. Department of Education, Institute for Education Sciences’ What Works Clearinghouse
• The U.S. Department of Labor’s Clearinghouse for Labor and Evaluation Research (CLEAR)
• The County Health Rankings and Roadmaps’ What Works for Health Clearinghouse
• Blueprints for Healthy Youth Development
• The Department of Justice’s Crimesolutions.gov
• Arnold Ventures’ Social Programs That Work

Where can I find funding or other resources to support evaluation?

Many federal agencies, philanthropic organizations, and research institutions offer opportunities to apply for funding or other resources, including:

• J-PAL State and Local Innovation Initiative which provides technical support, flexible funding and connections with their network of leading academic researchers.
• Arnold Ventures which funds randomized control trials focused on criminal justice, social programs whose delivery will be funded by government or other entities, and “moving the needle” on major social problems. Programs eligible for expansion under the Moving the Needle initiative are included here.
• Behavioral Insights Team which helps cities boost their impact through the application of behavioral science and rigorous, low-cost evaluation techniques.
• Coalition for Evidence-Based Policy’s Help Desk which offers brief expert advice, without charge in the areas of sponsoring or carrying out rigorous evaluations of program effectiveness, assessing existing evaluation studies, and developing evidence-based provisions for government solicitations and policy documents.

I need to contract with an evaluator for a grant – what should I do?

If you have the funding and would like to find an external party to conduct your evaluation, you’ll likely need to develop a request for proposals (RFP) The RFP should include a clear description of your evaluation needs as well as a clear description of what is expected of submitted proposals. Overall, it should contain three key parts:

1. Purpose: Description of the overall program that is being evaluated
2. Specifications: A detailed description of the evaluation services you are seeking
3. Evaluation Criteria: How each criterion noted in the “Specifications” section will be weighed

For more information, see the National Association of Counties Guide to Developing an RFP

How do I know if an evaluation is right for my program?

Below we highlight some important questions to ask yourself before implementing an evaluation.

Are there enough resources for everyone?

If you want to conduct a rigorous evaluation that involves the distribution of a service or good that has proven beneficial, it is not ethical to withhold this service or good if there is enough for everyone. In this case, a rigorous evaluation would not be appropriate.

Does the size of your program lend itself to being evaluated?

Small programs such as the distribution of a new communication material often are the easiest and most feasible to evaluate, particularly if you don’t have dedicated money or staff time for your evaluation. If your program has a large scope – either it will serve a large number of people, last a significant period of time, and/or represents a large amount of money – it is important to be mindful of what resources will be available to conduct an evaluation and whether it will be possible to sustain the evaluation through the life of the program. If you are implementing a large program and do not have financial and/or stakeholder support, a rigorous evaluation may not be possible

Would participating in the evaluation be burdensome or harmful?

Participant harm can be avoided by full engagement and transparency about the nature of the evaluation at the beginning of the process. Burden can be avoided by ensuring that the time being asked of participants is reasonable, will not significantly interrupt their normal lives and ensuring they are aware of all that will be asked of them before signing up for the evaluation. If you don’t feel you can ensure these things you should not conduct an evaluation.

For more information on determining whether an evaluation is appropriate or not, please see The Lab @ DC’s website on randomized evaluations.