Feedback is an innovative research, evaluation and information quality services provider, inspired by the challenges of sustainable development. By applying creativity, conscience, obsessive curiosity and uncompromising scientific rigor, we empower clients with the best possible empirical basis for taking action and making an impact.

0

Improving Evaluation Use: Policy Briefs

scout_trooper_utility_belt_templates_content_4

At FeedbackRA we take the response to our work personally. An evaluation exercise that fulfills a compliance imperative and decorates the bookshelf of some mid-level functionary (instead of bestowing upon the mid-level functionary powers of evidence to punch above her weight) erodes our sense of purpose, and we don’t like it. So we spend a fair amount of time thinking through how we can promote use. One of the more effective solutions (also feasible, unlike the emotionally charged calls for anonymous acts of intimidation – geeks have a dark side too) is composing a stand-alone policy brief that accompanies evaluation reports but that can also be distributed independently of the full evaluation.

A policy brief is a succinct summary of the implications of evaluation findings for decision-makers. The idea is to produce a single A4 page that focusses deliberations on evidence and guides decisions. The policy brief is structured as follows:

Conclusion
The brief leads with the objectives and evaluative statements or conclusions of the evaluation, as well as key lessons learned. To keep it short only the lessons with the mnost convincing body of substantiating evidence from the evaluation – in other words the findings representing the most reliable basis for taking action – are included.

Prioritize
This section of the brief highlights the most urgent calls to action indicated by the evaluation. These are often programming priorities but may be policy related if policy processes are already under way and policy level decisions are immanent.

Formulate
Policy recommendations are presented in this section. The intention of the policy brief is to facilitate evaluation use at levels of decision making beyond programme implementation. The crafting of this section is especially important. It must contain reasonable, unambiguous and evidence based recommendations that are not only convincing to the audience but are also posited by the evaluation team with a high degree of certainty.

Fund
If the evaluation provided evidence for either the most cost-effective spend, or ommissions in the programme design that would enhance effectiveness, then funding recommendations are approrpiate and are presented in this section of the policy brief.

Measure
An evaluation will frequently reveal gaps in the monitoring and evaluation framework developed for the intervention. If there is obvious adjustment to the M&E framework that would deliver significantly improved, utilisation focussed results, then those adjustments are proposed in this section of the policy brief.

Programme
Gaps in programme design, shortcomings in fidelity and the prerequisites for effective replication of the programme (should a finding on replication be required) are all addressed in a set of recommendations in this section.

Provided it is ruthlessly succinct, unambiguous, and firmly grounded in the stronger evidence provided by the evaluation, a policy brief is an indispensable weapon in the arsenal of the lowly evaluation stormtrooper bravely doing his daily duty at the frontline of the utility wars.

0

UNICEF Publication on Equity Focussed Evaluations

unicef1

A sharper focus on equity in human development is gathering momentum at the international level, supported by United Nations reports and strategies as well as by independent analysis. Increasingly national policies and international alliances are focusing on achieving equitable development results. This emphasis poses important challenges and offers novel opportunities to evaluation.

“How can one strengthen the capacity of Governments, organizations and communities to evaluate the effect of interventions on equitable outcomes for marginalized populations? What are the evaluation questions to assess interventions are relevant and are having an impact in decreasing inequity, are achieving equitable results, and are efficient and sustainable? What are the methodological implications in designing, conducting, managing and using Equity-focused evaluations?”

 The new publication “How to design and manage Equity-focused evaluations” represents a first attempt to address these questions. The document starts by defining equity, why equity matters and why equity is so urgent now. It then presents what an Equity-focused evaluation is, explaining what its purpose should be and potential challenges in its promotion and implementation. The second part of the document explains how to manage Equity-focused evaluations, presenting the key issues to take into account when preparing for the Equity-focused evaluations and developing the Terms of Reference, including presenting potential equity-focused evaluation questions; how to design the evaluation, including identifying the appropriate evaluation framework, evaluation design and appropriate methods to collect data; and how to ensure the evaluation is used. The document also addresses how to conduct Equity-focused evaluations under real-world constraints. Last but not least, eight case studies are included to illustrate how evaluations supported by UNICEF have addressed equity-focused issues.

 The new publication is available, free of charge, at  

http://www.mymande.org/?q=virtual  .

 An introductory webinar with the authors, and a PowerPoint presentation, are also available at http://www.mymande.org/?q=how_to_design_implement_and_use_evaluations

1

OECD Better Life Index

Data_vis

For those people who are interested in Data visualization, and measures of quality of life – Here is a neat resource and a truly elegant example of what can be done with data.

What’s Your Better Life Index?

Your Better Life Index is designed to let you visualize and compare some of the key factors – like education, housing, environment, and so on – that contribute to well-being in OECD countries. It’s an interactive tool that allows you to see how countries perform according to the importance you give to each of 11 topics that make for a better life. Unfortunately no information is available about South Africa, but I still think it is worth a visit!

I also share some other information about “Beautiful Information” and Data visualization here