At FAI, we’re big advocates for data. Why? Because you can’t make good policy without data. Data can be collected in many ways and come in many forms: transaction records, panel surveys, financial diaries, or field experiment results. We get excited about the opportunity to collect or analyze data about the financial behavior of poor households.
While we often lament the lack of data, it is equally important to remember that quality of data is just as important. Poor quality data leads to decisions just as bad, if not worse, than no data. Two recent posts on data quality are a good reminder of the lingering problem of data quality even where data does exist.
In a new podcast from EconTalk, Morton Jervin of Simon Fraser University discusses how the quality of data coming out of many African countries makes the many claims based on that data suspect. He argues that the data used to calculate a country’s income, growth, and population is not accurate, and therefore measures of a country’s GDP and growth rate are not reliable indicators of a country’s development. Since the quality of data varies from country to country and within a country across time, attempting to quantitatively represent the progress of African nations is essentially impossible. Jervin elaborates upon these ideas in his forthcoming book, “Poor Numbers: How We Are Misled African Development Statistics and What to Do About It.”
In a related blog post, Matthew Yglesias of Slate Magazine details how the U.S. is not exempt from issues relating to the use of unreliable data. He explains that all too often macroeconomic analyses are based upon shoddy numbers which renders even the smartest and most elaborate calculations meaningless. These misrepresentations are especially egregious when they leave the theoretical realm and are used to inform policy decisions.
The problem of poor quality data is one of the reasons that we’re so excited about the US Financial Diaries project. The data available on the financial lives of households below the median income in the U.S. is very poor today (where it does exist). The financial diaries approach will give us the kind of high quality data needed to make better policy decisions.
Update: I've just seen this new post from Berk Ozler on the Development Impact blog that goes far deeper into the issue of bad data--specifically how skeptical we should be when looking at self-reported data. I should note that the financial diaries methodology being used in the US Financial Diaries project is specifically designed to yield better data from surveys of household spending, as Daryl Collins describes in a chapter in Jonathan Morduch's new edited volume, Banking the World.
Berk also points to a special issue of the Journal of Development Economics from last year that features more than a dozen papers on how to improve data quality in development research. So much to blog!