If you only read one critique of the recent microfinance impact "statement," it should be Chris Dunford’s over at Freedom from Hunger. We’ve taken the liberty of excerpting our favorite parts for you and explaining why exactly we agree.
First, Chris says of Freedom From Hunger’s own experience with serious impact research: “The results…have validated some of our claims and failed to validate others. We are challenged to embrace the revealed weaknesses and to reflect with our practitioner partners and take action collectively to make important improvements in our products and services.”
We wholeheartedly agree that taking evaluation more seriously can help MFIs improve what they’re doing. We’ve pointed to BASIX as another good example of an organization that has used evaluations as a powerful force for constructive change in the way it offers financial services to the poor.
“The recent research studies in India and the Philippines seem to conform to best practices of credible impact research, so let’s accept the results for what they are, which are mostly positive and realistic.”
Yes. Microfinance is not the answer to ending poverty as we know it—nor should it be. We wouldn’t expect such results from education reform or maternal mortality programs. In the words of the impact study authors themselves, microfinance is delivering “exactly what a successful new financial product is supposed deliver—allowing people to make large purchases that they would not have been able to otherwise.”
“Stories are observations or data points like any other. But what are their limitations? We should ask some questions about them. Are we hearing the whole story of this individual or that family? What else is happening in her or his life? An advantage of the financial diaries and similar “story” collection methods is that we are more likely to capture the whole complex story by more in-depth and comprehensive observation and questioning than we do through either client interviews for fundraising communications or quantitative surveys with highly structured questions and response choices.”
Sure, stories can be data points, but the key to any kind of rigorous research is to let the data tell the story—not to craft a story from the data. Selecting anecdotes from client interviews to support your narrative is not research. We do believe in the financial diaries methodology, which is a mixed-research methodology that is systematic in data collection and, in its purest form, allows for statistical analysis. We also believe that quantitative surveys with structured questions can be invaluable if piloted and well-executed, as our partner organization, Innovations for Poverty Action, does.
And finally, our favorite paragraph from Chris’s piece:
“Everyone, let’s get real here! The research to date is not telling us to abandon microfinance but to learn from experience and improve products, systems and practice. The recent studies both validate some claims and reveal weaknesses in others. The results, and the various public responses to these results, remind us to keep on innovating and learning how to do better microfinance, better research, better marketing communications and better journalism.”