Some folks – perhaps only stats nerds like yours truly – noticed this item in the press last week: “Second-Quarter G.D.P. Revised Sharply Higher … Government statisticians gave the American economy a lift Thursday when they sharply revised their calculation of the nation’s second-quarter growth to an annual rate of 2.5 percent, up from an initial estimate of 1.7 percent.”
(Source)
What is striking about this huge revision – the new estimate means that the economy was growing almost 50% faster than initially estimated – is not so much the statistical work itself, but how the media, the financial markets, and public read them. The economic numbers are only approximations. Indeed, this 0.8 point revision is less than the average revision made on quarterly growth rates over the last almost 30 years. The problem is not with the statisticians working to estimate these numbers; they are top-notch. The problem is the non-statisticians who fail to appreciate how noisy the data are and who make strong claims and key decisions based on small variations that experienced researchers treat with appropriate distance.
(As a footnote to this post, I ask why two scholars recently dismissed economics as a science on the grounds that economists don’t predict the future – an odd argument.)
Number Appreciation
The vast bulk of economic data is generated by surveys, all of which have considerable “noise,” – i.e., error. Other economic data may come from sources such as tax filings, payroll records, financial transactions reports, corporate statements, and shipping manifests. Each of these has its own set of accuracy issues (as business scandals often show). But surveys – those same sorts of surveys asking people their opinions about foreign policy, whether they plan to buy a car, or if they voted – are the key source.
The University of Michigan Survey of Consumer Confidence generates one of the numbers that moves Wall Street up or down. While done quite rigorously (see here), the survey drawz on about only 500 respondents a month who are reached by telephone, an increasingly frustrating experience.
The Bureau of Labor Statistics, a great center of scientific expertise, is responsible for generating the estimates of how many new jobs are created each month. It is based on a survey of business “establishments,” massaged by all sorts of models of businesses’ “births and deaths.” The BLS appears to get pretty good cooperation from a sample of major companies that send over their payroll information, but the numbers are vulnerable to that cooperation and to the reporting practices inside the companies. And the BLS notoriously has a hard time knowing about and keeping track of small companies that pop up and disappear. Then there are messy issues such as who is officially employed, who is a “contract” worker, and all those people who work off the books.
The unemployment rate comes from the BLS’s huge and comprehensive Current Population Surveys conducted for it by the Census Bureau. Census staff systematically interview 60,000 households a month in a careful procedure. There are, nonetheless, many vagaries. To be considered unemployed, respondents need to be not working – including not working “under the table” – and say they are actively looking for work. One would hope that the percentage of respondents who report “incorrectly” would always be a tiny and random subset. But that is unlikely. Economic strains could lead more people to work off the books and more people to claim they are looking ir order to hold in to unemployment insurance as long as possible.
I have only touched on the many ways that economic data can be distorted. But the lesson to draw is not to discount the statistics. It is to take them with appropriate caution. (Recall during the 2012 presidential election how each month’s nudge up or down in the unemployment rate was treated as a political tidal wave.) Yes, the unemployment rate is still much higher than it was before 2009 and, yes, the rate for blacks is about double that of whites. But obsessing about tenths of a point in that rate or about tens of thousands of new jobs is failing to appreciate the how the numbers are estimated.
Prediction – Bah!
The second science note is a brief observation. Two scholars recently writing in the New York Times claimed that economics was not a science because (most) economists did not predict the recent economic shocks. If economics cannot predict the future, they claimed, it cannot be a science. I was stunned by the obtuseness of this argument.
First, generally predicting human behavior is actually easy. City planners and traffic engineers predict quite accurately how many people are going to drive on which roads on which days. Demographers predict with considerable preciseness how many people are going to die and how many will be born in a given year. Even economists can predict what will happen if the price of a good (most goods) goes up or down significantly; demand will move. What the complainants about prediction complain about is the inability of social scientists to predict the rare and unusual event. Duh. That is true almost by definition. It is as if we decided that earthquake scientists were not real scientists because they did not predict the great tsunami of 2011.
Second, most disciplines that we recognize as sciences do not make predictions. Evolutionary biologists do not predict what creatures will look like in the future. Geologists aren’t predicting rock formations in the next millennium. Medical researchers at best give people odds on surviving certain diseases; they avoid actual predictions.
What sciences do when they in fact predict is not predict the future, but predict what the result of a piece of research would be if a theory being tested is correct. For example, things will blow up if you combine chemical A and B; groups will make different decisions if the gender mix is altered. Otherwise, science is not about predicting; it is about explaining.
MADE IN AMERICA 