How we exposed misconceptions around microfinance

Microfinance, now more commonly referred to as financial inclusion, is a cornerstone concept in helping people in emerging economies to access finance.

Beginning in the 1970s and rising to prominence across the 1990s, the microfinance model works by making small loans directly to people who are typically excluded from traditional banking services. Its aim was to empower those most in need (often women) in the poorest nations.

As an idea, it was ambitious in scope and seemed win-win; reaching beyond traditional gatekeepers of finance and offering the poorest the opportunity to help themselves. As it grew in popularity, it became more firmly entrenched in policy circles as an untouchable ideal, supporting poverty alleviation and women’s empowerment.

The project

But Maren Duvendack, Professor of Evaluation in Economics at the University of East Anglia, is challenging this perception in ground-breaking research. Her paper, “High Noon for Microfinance Impact Evaluations: Re-investigating the Evidence from Bangladesh”, along with a subsequent impact case study, debunked the received wisdom around microfinance and its claim to help alleviate poverty and empower women.

“My research has helped change the views of practitioners, academics and policymakers that microfinance has beneficial impacts, by challenging the quality of the underlying evidence base,” says Prof Duvendack. “Failure to replicate seminal studies claiming positive impacts of microfinance suggests that we need to produce more rigorous evidence and encourage more transparency in the research process.

Adopting an open approach

Open research practices made Prof Duvendack’s critical discoveries possible as they allowed her to demonstrate that the evidence base supporting microfinance – in the form of foundational, quantitative, data-driven studies – simply could not be replicated. This uncovered a lack of scientific rigour, a blinkered focus on metrics and, essentially, an entire enterprise built on “foundations of sand”.

“The microfinance (now financial inclusion) sector is one of the fastest-growing areas in international development, promoted further by the Sustainable Development Goals’ (SDGs 1 and 5) focus on expanding access to financial services,” Prof Duvendack explains.

“The expectation underlying microfinance is that greater access to financial services will create poverty-alleviating and empowering effects. This makes it crucially important to ensure the validity of the evidence base that is used to verify the claims that microfinance is the silver bullet to improving the lives of the poor.” 
Prof Duvendack has long championed open science, promoting the use of open research methods in the form of replication studies involving collaborative efforts and the sharing of data and code to facilitate transparency of the research process. This is particularly important for microfinance because of its bold claims to alleviate poverty and empower women.

Because replication studies are set in a wider, collaborative framework they can prompt engagement between researchers and encourage further study, which in turn deepens and drives insights into real-world challenges.

“The value of collaboration is being able to tap into the knowledge and experience of a variety of colleagues from a wide range of academic disciplines within and beyond one’s institution,” Prof Duvendack says. “This enables creativity and innovation to thrive, and broadens one’s network, making the research process an enriching experience.”

Additionally, any misconceptions can be weeded out before they become ingrained into practice and end up influencing wider policy and public opinion, as with microfinance.

Overcoming challenges

Prof Duvendack nonetheless faced resistance when trying to encourage peers to adopt replication.

“Fellow economists are often tardy to respond to requests for sharing of data and code, and no response at all is not uncommon,” she says. “Frequently, excuses of having lost data and code are put forward to avoid having to share information, or sometimes excuses relate to citing confidentiality of the data source as a reason not to share it.”

Economics is a discipline where journals favour new research – so engaging in replication was felt, by some, to be covering old ground. Relatively few replication studies have been published in economics; Prof Duvendack’s research has also revealed that only 16 of the top 50 economics journals have published three or more replication studies in the last 60 years.

“There are disincentives for replication in economics, caused by distorted researcher and publisher incentive systems,” she explains. “Since replication seemingly involves repetition, it is almost by definition not a rewarding activity.”
 

"Many replication initiatives are now under way, and the increasing availability of data and code, technological innovations in the allocation of journal space, and societal factors that affect ‘tastes’ for replication research are all likely to expand the use of replications in the future."

Progress may only be possible with full transparency, but sometimes the culture takes time to catch up. In Prof Duvendack’s case there were also simple, practical issues associated with replication.

“Replicating the work of others requires persistence and patience” she says. “You almost need the mindset of a detective, especially when datasets and code are incomplete, sprawling, impenetrable, and poorly organised – or all of these at once.

“Analytical approaches can also be murky, with descriptions of methods clouded by jargon. Unpicking the methodologies and cleaning up raw data can be extremely time-consuming, so advocating replication is ever more important to instil good research practices in the research community.” 

Outcomes and results

Thanks to her adoption of open research and championing methods such as collaborative working, sharing of data and code along with instructions for how to use them as a fresh approach, Prof Duvendack helped to bust the myths around microfinance, reframe the debate around it and, in a wider sense, affirm replication studies as important – and often overlooked – scientific tools. This has led to far-reaching, real-world change.

Prof Duvendack showed that the political preference for “hard” evidence had steered the focus towards flawed quantitative data on microfinance. This sidelined the nuance, context and insight that including qualitative data as part of a mixed-method approach provides. Microfinance was then unmasked as making little difference to the lives of the poor.

Instead of enabling people to set up thriving businesses, many enterprises failed because those encouraged to borrow often lacked financial literacy skills and instead used the loans for consumption.

“Because of some of my work on microfinance, the debates in the sector have shifted from microfinance to financial inclusion,” says Prof Duvendack. “This has triggered some soul-searching as to how best to improve the methodological rigour of microfinance impact studies and thus best serve poor people with appropriate financial products.”

Prof Duvendack and Prof W Robert Reed from the University of Canterbury in New Zealand have gone on to set up The Replication Network, a world first for economics. The site is a hub for collaboration in open research, allowing for the sharing of news and developments, networking and blog hosting where studies are presented and discussed.

With an emerging trend of replication demanded by journal editors, funders and policymakers, research transparency is becoming increasingly the norm. The outcome is an improvement in research quality and more scientific rigour.

“Replication is no panacea, but it is a useful tool for assessing the reliability and validity of empirical results and I hope to further the cause with my work,” says Prof Duvendack.

Benefits of open research

Open research methods can drive continuous improvement of scientific standards. Research has shown that only a small percentage of published scientific research can be effectively reproduced. Data errors, data mining and researchers losing sight of wider themes when searching for statistical significance remain serious issues.

These issues can be hugely reduced with an open research-led focus on reproducibility. “I’m hopeful that we’re seeing a shift in economics and its attitudes towards replication,” says Prof Duvendack. “Many replication initiatives are now underway, and the increasing availability of data and code, technological innovations in the allocation of journal space, and societal factors that affect ‘tastes’ for replication research are all likely to expand the use of replications in the future.”

Quantitative data should only be elevated to status of fact if it is verifiable and made available openly. And, thanks in part to Prof Duvendack’s ground-breaking work, the landscape in economics is beginning to shift, with economic journals increasingly requiring studies to provide data and code to allow replication before they will be published. In fact, Prof Duvendack is an editor of the Journal of Comments and Replication in Economics, a unique journal focusing on the publication of replication studies in economics.

With data and code in the public domain, more researchers will be able to rerun studies and verify their findings in the future. As academia adopts this new culture, there’s less angst and protective attitudes over results. New collaborative practices, increased networking and co-operation between staff and institutions are emerging.

For more on the life-changing research taking place at UEA, follow @UEAResearch on Twitter.