For those of you who couldn’t attend Wednesday’s event, “Microfinance's Social Impact: Cutting through the Hype,” sponsored by the Microfinance Club of New York and hosted by FAI, here’s a recap.
The panel featured FAI’s Jonathan Morduch, David Roodman, a senior fellow at the Center for Global Development, Chris Dunford, President of Freedom from Hunger, and Jody Rasch from Moody’s Investors Service.
We were happy that the moderator started the event by making the important distinction between social performance measurement and impact evaluation. Social performance measurement seeks to answer the questions of how MFI clients are doing, and how MFIs can help serve those clients better. Social impact evaluation, which is the focus of much of FAI’s research, tries to answer the question of what would have happened to people in the absence of microfinance.
Jonathan Morduch opened by posing the basic question of “Why evaluate?” He pointed out that at the beginning, economic theory indicated big potential gains from microcredit, but real evidence was missing. There were lots of success stories floating around, but what was needed was a study of microfinance clients who weren’t selected to be inspiring, but to provide real data on how microfinance was or wasn’t affecting their lives.
Morduch pointed out that while the current impact studies reveal smaller impacts than we might have hoped for; there is no cause for panic. He laid out some evidence from his financial diaries research that suggests that the difference in people’s lives with and without microfinance is probably smaller that we imagined, due to all the informal financial services they employ. The real questions that impact evaluations can help answer are not whether microfinance is transformative (probably not) but who is being helped by it, why it’s working in those cases, and how we can make it work better for more people.
Morduch’s bottom line: Impact studies can be powerful in creating a new agenda for microfinance.
Chris Dunford of Freedom from Hunger spoke next and offered a practitioner’s perspective on impact evaluations, and why more MFIs aren’t committed to rigorous evaluation. He logged the complaints that a) impact evaluations are expensive, b) they take a long time, and c) at the end of it all it’s often hard to figure out what (if anything) you’ve learned. On the other hand, Dunford acknowledged that impact evaluations are necessary. In his words, it’s “the only way we collect data from clients that eliminates alternative explanations for results.” He says that while impact evaluations are necessary, they are not enough.
Dunford’s opinion is that a mix of stories and impact evaluations provides a much richer mix of information. Through story collection you discover things that you wouldn’t discover in an impact evaluation, because you have more flexibility to discuss and probe and ultimately reveal information that wouldn’t have been captured through the rigidity of an impact evaluation survey.
Dunford’s bottom line: Impact evaluations are necessary, but they are not enough; you also need to collect stories that provide a richer mix of information.
Jody Rasch from Moody’s Investors Service expanded the discussion with a presentation on the research Moody’s is doing with an eye toward developing a social performance rating methodology. He said this research was spurred by the questions he asked himself: “Why wouldn’t microfinance work? Why do we find in some impact studies that while some people do well, some people don’t?” He suspected that in some cases this had to do with a range of broader, MFI-specific factors, like collection practices, interest rates, and other performance issues.
So why not measure the “good” MFIs compared to the not-so-good ones to see why there is a difference in their clients’ lives? This leads, of course, to problems identifying which are the “good” MFIs, and this is where social performance measurement becomes important. Rasch made the point that while “impact studies show what has been done, they don’t show what should be done – this is where we can start looking to social performance.”
Rasch’s bottom line: While impact studies show what has been done, they don’t show what should be done – and here social performance measurement, when done well, can help.
David Roodman closed the evening by saying that impact evaluations are valuable, but that they’re not all we should rely on. He pointed out that it’s often hard to discern whether people are better off because they’re borrowing, or whether they’re borrowing because they’re better off – saying randomized control trials can help identify the causal chain.
Roodman discussed his three notions of success in microfinance (explored in more detail on his blog). The first is whether microfinance can be shown to reduce poverty, and the jury is still out on this one. The second stems from the notion of development as freedom – and specifically the question of whether access to financial services leads to more agency for microfinance clients. He referenced the work of Stuart Rutherford and the financial diaries as a good source of data on this. The third is the notion of development as economic transformation. Roodman said that in his view, microfinance has been very effective in this – in creating institutions that enrich the economic fabric of society.
Roodman’s bottom line: Impact evaluations are valuable, but that they’re not all we should rely on to measure the success of microfinance.