Skip to Content
  • Intro
  • Key Research Areas
  • Training
  • Events
  • People

Impact Evaluation

Impact evaluation is a growing area of research strength and teaching capacity in the School of International Development. We have been doing evaluation since our inception in the late 1960s, in our undergraduate, postgraduate, research and consultancy activities, including traditional cost-benefit studies, monitoring and evaluation (M&E), and qualitative as well as quantitative research. However, the current wave of enthusiasm for impact evaluation in international development emphasises quasi-experimental and experimental research designs, micro-econometric analyses, qualitative and mixed methods, and systematic reviews, and new and renewed modes of M&E. Staff of this research group have been and are involved in all these areas. A dynamic multidisciplinary team in DEV is currently driving the growing field of impact evaluation, systematic reviews, meta-analysis and experimental games. Together these activities and capacities provide an exciting environment for teaching and study of impact evaluation and a wealth of experience for research and consultancy.

This group has also been very active in terms of teaching. It organizes a unique MSc Impact Evaluation for International Development to address the evident need for impact evaluation capacity. The MSc offers familiarisation with and skills in the basics of modern evidence-based policy-making and impact evaluation, including the contexts and practices of evaluation, research design and data production for evaluation, and basic and more advanced methods of quantitative and qualitative analysis. Graduates of the MSc have joined ODI, 3ie and other well-known institutions engaging in impact evaluation.

In addition, this team organizes a professional short course in Impact Evaluation for Evidence-Based Policy in Development. This offers practical impact evaluation training to individuals working in international agencies, governments in developing and developed countries, think-tanks, NGOs and other donor organisations which need to understand the methods used in evidence-based policy making in order to evaluate and justify continued public spending on particular projects and programmes. Moreover, the short course is being offered overseas to build evaluation capacity in particular in developing countries conducts and has been tailored to meet the demands of e.g. Uganda civil servants, NGO officials in Ghana etc.

We are also now offering a second short course on impact evaluation: Beyond Surveys and Experiments - Other Approaches to Impact Evaluation.

Researchers from the UEA have joined the Centre for Development Impact (CDI), an exciting initiative that contributes to learning and innovation in the field of impact evaluation.

 

Impact Evaluation

Year: 2015

Title: Evaluation of the Chhattisgarh Tribal Development Programme

Funder: International Fund for Agricultural Development (IFAD), Status: Live, Dev researchers: Bereket KebedeMaren Duvendack

Using a quantitative approach, this project evaluates the impact of the Chhattisgarh Tribal Development Programme in the state of Chhattisgarh, India. The programme aims to empower tribal people—who are among the poorest people in India—to participate in their own development through local self-government. Although the programme targets both tribal and non-tribal populations, the tribal population represents the largest share. Dr Kebede and Dr Duvendack also provide inputs to the second edition of the Independent Office of Evaluation of IFAD (IOE) evaluation manual. 

Year: 2014-2015

Title: Evaluation of the Results Based Aid in Education Sector Rwanda

Funder: DFID, Status: Live, Dev researchers: Paul ClistBereket Kebede

This project evaluates the implementation of the results-based aid (RBA) three-year pilot in Rwandan education (2012-2014). Using a mixed-methods approach, the project provides insights into progress – and barriers to progress – with respect to completion and teachers’ competence in English. Combining quantitative modelling with qualitative fieldwork at district, sector, and school-level, the project examines what happens ‘on the ground’ in relation to the agreed RBA results.

Year: 2014-2015

Title: Technical support to three ex-post impact evaluations using mixed methods approaches across three countries: Laos, Ghana and Cambodia

Funder: International Fund for Agricultural Development (IFAD), Status: Live, Dev researchers: Bereket KebedeMaren DuvendackJennifer Leavy

This project aims to enhance the delivery of public services for poor rural people in Cambodia, Laos, and Ghana. Using a mixed-methods approach, four IFAD-supported projects are evaluated with a specific focus on women’s empowerment, movements out of poverty, and economic resilience and adaptive capacity. Lessons learnt from the evaluation are extracted to strengthen public institutions’ impact evaluation and monitoring and evaluation capacity.

Year: 2014

Title: Supporting Evaluation Capacity in Government and Civil Society in Uganda

Funder: GIZ, Status: Complete, Dev researchers: Maren Duvendack

To enhance the use of evaluation in policy-making in Uganda, this project developed national evaluation standards in collaboration with the Evaluation Sub-Committee (ESC) and the Uganda Evaluation Association (UEA) consisting of representatives of the Ministry of Finance, Planning and Economic Development, the Parliament of Uganda, and a number of NGOs and private sector actors. A short-term training course was provided for ministries, departments, agencies, Parliament and civil society.

Year: 2014

Title: Evaluating Development Impact Bonds

Funder: DFID, Status: Complete, Dev researchers: Paul Clist

Being a new aid modality, Development Impact Bonds (DIBs) pose particular challenges for evaluation. Grounded on economic theory, this study presents concepts for understanding how data from evaluation of individual DIBs can be synthesised into an overall evidence base. Based on a review of relevant grey and academic literature and consultation of a number of key informants, a framework to use for evaluation of DIBs is presented.

Year: 2013-2014

Title: Valuing Progress: Gates Foundation: Development Progress II

Funder: Bill & Melinda Gates Foundation, Status: Complete, Dev researchers: Laura Camfield

The project explored ways of measuring development outcomes that are based on poor people's own values and priorities. Background research was undertaken on issues relating to Valuing Progress, including evaluation of methodologies for measuring wellbeing and constructing composite measures. A research proposal was developed to seek extension of Valuing Progress as a pilot project. The outputs of the project will be published in a book chapter in ‘Mixed methods research in poverty and vulnerability: sharing ideas and learning lessons.’ (Roelen, K. & Camfield, L. 2015) as well as in an Overseas Development Institute working paper. 

Year: 2009-2010

Title: Impact of the irrigation improvement component of the Agricultural Sector Programme Loan (ASPL)

Funder: 3IE, Status: Complete, Dev researchers: Richard Palmer Jones

This observational study analysed the impact of the Mae Lao Irrigation Project in Thailand. Using a quasi-experimental design, farm plots within the irrigation project were compared to farm plots in neighbouring irrigation schemes not subject to the intervention. The study design exploited variation within irrigation schemes, between on-scheme irrigated and irrigable areas and off-scheme unirrigated areas, and between scheme variation in a cross-section survey. The full report can be found here.

 

Advisory

Year: 2015-2017

Title: Child Development Grant (CDG) impact evaluation

Funder: ITAD CDG, Status: Live, Dev researchers: Laura Camfield

By providing peer review support, this project strengthens the Child Development Grant (CDG) impact evaluation process. Dr Camfield supports the Information Technology and Agricultural Development (ITAD) evaluation team to effectively integrate the three evaluation components: randomised controlled trial, process evaluation, and qualitative evaluation in a robust mixed methods approach. 

Year: 2015-2016

Development of guide on how to apply Qualitative Comparative Analysis (QCA)

Funder: Swedish Expert Group for Aid Studies, Status: Live, Dev researchers: Laura Camfield

The Qualitative Comparative Analysis (QCA) is a method particularly well suited for systematic and rigorous comparisons and synthesis of information over a limited number of cases. Dr. Camfield is a member of the reference group convened by the Swedish Expert Group for Aid Studies (EBA) to develop a step-by-step guide on how to apply and quality-assure the application of QCA to real-life development evaluation, indicating the common mistakes and challenges for each step.

Project website

Year: 2015-2016

Title: Midline and Endline Study for Save the Children Girls’ Education Project

Funder: Save the Children, Status: Live, Dev researchers: Laura Camfield, Maren Duvendack

Using quantitative and qualitative research methods, this project conducts the midline and endline studies for Save the Children Girls’ Education Project in Mozambique. With a focus on the most marginalised girls’ school attendance and learning outcomes in the provinces of Gaza, Manica, and Tete, the components to be evaluated include: the project design and implementation, the impact on retention and learning, value for money, sustainability, and success in leveraging additional interest and investment.

Year: 2015

Title: Evaluation to estimate the impact of mNutrition

Funder: DFID, Status: Live, Dev researchers: Laura Camfield

mNutrition is a global initiative supported by DFID, organized by GSMA, and implemented by in-country Mobile Network Operators to use mobile technology to improve the health and nutritional status of children and adults in the developing world. The potential to utilize mobile technology to change attitudes, knowledge, behaviours and practices around health and agriculture for improved nutrition status has been recognized for some time, but to date there have been no rigorous evaluations of m-services at scale. A consortium of researchers led by IDS will conduct a rigorous evaluation to estimate the impact of mNutrition on adults and children. Dr Camfield provides advisory on the qualitative components of the study.

Year: 2015

Title: EU Expert Workshop on Research and Innovation Evaluation Methodologies 

Funder: European Commission, Status: Live, Dev researchers: Laura Camfield

The legal base of the current EU Framework Programme for Research and Innovation

(Horizon 2020) has set out ambitious and challenging evaluation requirements including the assessment of longer-term impacts and EU added-value of the Programme. A workshop with leading evaluation experts was organised in order to map the most appropriate evaluation methodologies for measurement of macroeconomic and wider societal impacts of national and/or European research and innovation funding, as well as to define how these methodologies could be best applied in the area of research and innovation.

Year: 2015

Title: BBC Media Action

Funder: BBC Media Action Global Grant, Status: Live, Dev researchers: Laura Camfield

The BBC Media Action Global Grant 2012-2016 targets to reach 200 million people in 14 countries living in poverty—especially women and girls—through mass media. By conveying knowledge about governance, health, and humanitarian crises, the aim is to enable people to make informed choices, take actions for their own development, and influence decision makers and hold them to account. As a member of the BBC Media Action Global Grant advisory group, Dr Camfield provides advisory on qualitative and mixed research methods in the evaluation of the BBC Global Grant.

Year: 2015

Title: Evaluation of Poverty and Peace (POVPEACE) programme

Funder: The Research Council of Norway, Status: Live, Dev researchers: Laura Camfield

The Norwegian government’s Poverty and Peace research programme (PovPeace) 2005–2013 focused on research on international poverty issues and research relating to war, peace, and development. As member of a group of experts advising on the evaluation of PovPeace, Dr Camfield provides advisory and support provides advisory on qualitative and mixed research methods for the company Oxford Research that conducts the evaluation.

Year: 2013

Title: Building Young Futures Longitudinal Survey Small-Scale Impact Assessment

Funder: UNICEF and Barclays, Status: Ongoing, Dev researchers: Laura Camfield

Applying quantitative and qualitative methods, this longitudinal study assessed the impact of the UNICEF UK and Barclays partnership programme “Building Young Futures”. With the use of surveys, focus group discussions and one-to-one interviews, the study determined in each programme country—Brazil, Egypt, India, Pakistan, Uganda, and Zambia—how young people perceived themselves and the progress they had made in becoming economically empowered. Dr Camfield joined as an advisor mid-way through the setting up of the qualitative research to support the quality of data generation, analysis and reporting (for example, through collaboratively developing guidelines).

Systematic Reviews

Year: 2014-2015

ESRC-DFID joint Fund for Poverty Alleviation Research Evidence Synthesis Research Award (ESRA)

Funder: ESRC-DFID, Status: Live, Dev researchers: Laura Camfield, Maren Duvendack           

This project maps the methodologies used across the 115 ESRC-DFID Joint Scheme research projects, paying particular attention to research design, sampling, and methods, including the diversity of methods and the mixing of qualitative and quantitative, research ethics, data analysis and preparing data for archiving. The aim is to provide an overview of the research designs summarising the advantages and disadvantages of each, and to identify methodological gaps that could potentially be met by training or other resources, for example, are there methods that are not being used successfully? Are there methods that are not being used at all? 

Year: 2014-2015

Title: What policies and other interventions have been strongly associated with the translation of growth into reductions in income poverty?

Funder: DFID, Status: Live, Dev staff: Ed Anderson, Maren Duvendack, Lucio Esposito

This systematic review aims to establish whether any specific types of policies have a consistent positive or negative effect on the translation of growth into poverty reduction, and to explain any heterogeneity in the estimated effects of particular policies, in low and middle income countries. The research question this systematic review addresses is: “What policies and other interventions have been strongly associated with the translation of growth into reductions in income poverty?” A combination of meta-analysis and meta-regression is applied to studies that use an appropriate counterfactual in assessing the impact on income inequality - namely ex-post quasi-experimental studies and ex-ante simulation studies.

Year: 2014-2015

Title: What policies and interventions have been strongly associated with reductions in in-country income inequality?

Funder: DFID, Status: Live: Dev researchers: Ed Anderson, Maren Duvendack, Lucio Esposito

This systematic review aims is to establish evidence whether any specific types of policies tend to reduce or increase income inequality on average, and to explain any heterogeneity in the estimated effects of particular policies on income inequality in low and middle income countries. The research question this systematic review addresses is: “What policies and interventions have been strongly associated with reductions in in-country income inequality?” A combination of meta-analysis and meta-regression is applied to studies that use an appropriate counterfactual in assessing the impact on income inequality - namely ex-post quasi-experimental studies and ex-ante simulation studies.

Year: 2011-2014

Title: The Effect of Microcredit on Women’s Control over Household Spending in Developing Countries: A Systematic Review

Funder: The Millennium Challenge Corporation of the United States Government, Status: Completed, Dev researchers: Maren Duvendack, Richard Palmer-Jones

This systematic review focused on the evidence on the effects of microcredit on women’s control over household spending in developing countries. The study answered to the research questions: 1) What does the impact evaluative evidence say about the causal relationship between microcredit and specific dimensions of women’s empowerment (women’s control over household spending)? 2) What are the mechanisms mediating this relationship?

Studies analysing the effects of microcredit schemes targeting poor women in low and middle income countries and studies giving evidence of addressing the attribution problem—either through randomised design, quasi-experimental matching, or regression analysis—were included. Studies that did not include analysis on microcredit and the effect on one or more dimensions (specified in main body of the report) of women’s control over household expenditures were excluded.

An initial number of 310 papers were selected for full text examination. 29 papers, corresponding to 25 unique studies, were retained for further analysis. The study found that although the microcredit evidence base is extensive, most studies are weak methodologically. From those studies deemed comparable and of minimum acceptable quality, the authors concluded that overall there is no evidence for an effect of microcredit on women’s control over household spending.

The full study and protocol are available at: http://www.campbellcollaboration.org/lib/project/178/

Year: 2011-2013

Title: Do Institutional Mechanisms for Water Resources Management Result in more Equitable, Sustainable and Efficient Allocation and use of Water Resources in Terms of Improved Resilience of Poor People to Floods and Droughts?

Funder: DFID, Status: Complete, Dev researchers: Virginia Hooper, Denis Da Silva, Mark Zeitoun, Bruce Lankford

The project compiled a systematic map to define the extent and focus of the literature on the following research question: ‘What factors determine the performance of institutional mechanisms for water resources management in developing countries in terms of delivering pro-poor outcomes, and supporting sustainable economic growth?’

After screening over 29,000 articles, only 38 papers were found to be relevant.  The systematic map concluded that the pool of reliable knowledge is small and that we lack the evidence to: a) confirm whether water resource management institutions are performing; and b) comprehend and manage the range of factors shaping that performance.

The outputs of the study included a systematic map database: http://www.environmentalevidence.org/wp-content/uploads/2014/07/CEE11-006.pdf and presentations at international conferences such as World Water Week (2012).

Year: 2010-2011

Title: What is the evidence of the impact of micro-credit on the incomes of poor people?

Funder: DFID, Status: Complete, Dev researchers: Richard Palmer-Jones, Lee Hooper, Nitya Rao, Maren Duvendack

Despite the apparent success and popularity of microfinance, no clear evidence exists that microfinance programmes have positive impacts. To address this gap, the systematic review focused on the research question ‘What is the evidence of the impact of microfinance on the well-being of poor people?’ Eleven academic databases, four microfinance aggregators, eight non-governmental (NGO) and aid organisation websites were searched. Additionally, bibliographies of reviewed books, journal articles, PhDs, and grey literature were consulted. Articles were screened reducing 2,643 items to 58 which were examined in detail. The outcomes of the review can be found here.

 

Professional Short Courses

UEA Dev currently offers two impact evaluation short courses for professionals.  Please follow the link for more information on topics, the teaching team and dates of the courses.

University Degree Programmes

The UEA School of International Development currently offers an MSc in Impact Evaluation for International Development

Bespoke Training

UEA Dev offers bespoke training, which is designed and tailored to meet training needs. Below are some examples of bespoke training projects that Dev researchers have facilitated. Any queries regarding impact related training please contact Maren Duvendack (m.duvendack@uea.ac.uk)

Year: 2014

Title: Evaluation Methodology Training for Analysts, Funder: BIS (Department for Business, Innovation and Skills), Status: Complete, Dev researchers: Maren Duvendack, Paul Clist, Jennifer Leavy. 

This Evaluation Methodology Training was designed for analysts employed at the Department for Business, Innovation and Skills (BIS). Modules included: matching Techniques, regression and difference in difference, regression discontinuity design, instrumental variables, panel data methods, meta-analysis (Systematic Reviews) and STATA.    

Year: 2013

Title: Evaluation Methodology Training for Analysts, Funder: BIS (Department for Business, Innovation and Skills), Status: Complete, Dev researchers: Maren Duvendack, Ed Anderson, Sunil Kumar 

A series of training on Evaluation Methodology for economists, statisticians, social researchers and analysts at the UK Government Department for Business Innovation and Skills (BIS) were carried out. Providing a common platform for departmental consistency in evaluation, the training focused on key methodological issues and cost-benefit analysis.

Year: 2013

Title: Impact Evaluation Training Ghana - Facilitation of Introduction to Impact Evaluation, Funder: Participatory Development Associates (PDA), Status: Complete, Dev researchers: Maren Duvendack

A 3-day introductory course on Impact Evaluation was conducted for programme managers, desk officers and field staff from NGOs, funding agencies, and national and international development organisations in Accra, Ghana. Using lectures, discussions, group work, and Stata data analysis demonstrations, the course covered among others the theory of change, understanding the evaluation context, sampling and power calculation, data collection strategies and management, and systematic reviews and meta-analysis.

Year: 2013

Title: Impact evaluation training of Trainers for GIZ Uganda, Funder: GIZ, Status:  Complete, Dev researchers: Maren Duvendack, Laura Camfield, Richard Palmer-Jones

The aim of this training of trainers (ToT) was to build a critical mass of trainers on evaluation in Uganda who could then roll-out a training programme among a larger target group consisting of, for example, administrators from the Office of the Prime Minister, project managers, public sector officials, the private sector, and civil society. The training covered quantitative and qualitative methods,  value-for-money (VFM) analysis and an introduction to different evaluation designs.

Upcoming Events

UEA Dev and IDS will be holding an impact evaluation event early next year in London, details to follow shortly.

People

Laura Camfield - Senior Lecturer 


 Paul Clist - Lecturer in Development Economics


Maren Duvendack - Lecturer in Development Economics


 Ben D'Exelle - Senior Lecturer


 Bereket Kebede - Senior Lecturer


 Jennifer Leavy - Senior Research Fellow


 Arjan Verschoor - Reader