Big TFP data mystery! (Probably solved!)

NOTE: Mystery probably resolved! See update below. Here was the original post, for posterity:

While recently complaining about the overselling of static-efficiency policies, I asserted that rich countries have all grown at about the same long-term rate, despite decade-long divergences. I was talking, of course, about Total Factor Productivity, which at long horizons should be determined by technology.

I had been under the impression that over the last three decades or so, the rich countries had all experienced similar rates of TFP growth. My source for that was the OECD's time-series on multifactor productivity (another name for TFP). Here is a chart of those OECD productivity numbers since 1985:

As you can see, most rich countries grew their TFP at the same average rate, consistent with the idea that TFP mostly measures technology in the long term, and that technology spreads rather easily between rich countries. A few countries, like Korea, Ireland, and Finland, did much better over this period, and a few countries, like Italy, Spain, and Portugal, lagged behind. But most rich countries were clustered along the same basic line. The U.S., UK, France, and Germany (highlighted on the graph) all stayed very close to each other.

But I now see that FRED has its own TFP numbers for various countries, taken from the Penn World Tables. And here's what happens when I plot the TFP numbers for the U.S., UK, France, and Germany over the same time period (1985-2011):

What??!!

The U.S. and UK lines match up as before, but the Germany and France lines are wildly, totally different! In fact, according to the Penn World Tables, Germany's TFP actually steadily declined from the mid-80s to 2011! 

What on Earth is going on here?? Obviously the two measurement methodologies are very different. So I tried to track down the source of the discrepancy, and I found some interesting stuff. 

First of all, it turns out that the Penn World Tables, currently assembled by a team of economists from UC Davis and the University of Groningen, have undergone substantial revisions to their methodology in recent years. They switched to a new growth accounting method developed by Francesco Caselli in the early 2000s (which I plan to study in detail when I get the chance). As Antonio, and Marek Jarociński pointed out in 2010, these revisions were enough to substantially change the results of all cross-country growth regressions. Simon Johnson, William Larson, Chris Papageorgiou, Arvind Subramanian criticized the new Penn methodology, and suggested possible changes. 

Meanwhile, the OECD methodology for calculating TFP has some questions surrounding it as well. To get TFP you need measures of labor and capital inputs. The OECD uses a pretty textbook method for doing this - simply stick in the raw estimates for the dollar values of labor and capital. But when they tried using another database called EU-KLEMS that tries to adjust for "quality" of inputs, they found totally different numbers.

I am not experienced enough in growth accounting to wade into these disputes in a substantive manner; it would take me at least a month of serious study to be able to say with any confidence which of these methodologies I believe most. The real takeaway here, though, is that TFP measurements are HIGHLY suspect, and will continue to be so for the foreseeable future.

That is bad news for most of modern macroeconomics, both on the growth theory and on the business cycle theory side of things. If differing methodologies for measuring labor and capital inputs diverge by this much, it means that any series you use probably has tons of stuff in it that it shouldn't have. That means that changes in the series at business-cycle frequencies - the good old TFP shocks of RBC models, which are also part of "kitchen sink" DSGE models like Smets-Wouters - are also unreliable. Basically, all those "shocks" are as likely as not to just be noise. That's probably true whether you compare across countries or look only at one country.

So this is a very pessimistic finding, and a huge challenge for the growth accounting field. Hopefully, a meeting of the brightest minds will get to the bottom of the problem and arrive at a consensus solution. If not, it means that any model that relies on measures of aggregate TFP, or factor inputs in general, is unreliable until the accounting problems are worked out.

Updates

Robert Inklaar of the University of Groningen contacted me and explained what was wrong! The most recent version of the Penn World Tables, version 8, did not take into account changes in averaged hours worked in some countries. Also, it used a Barro-Lee data source that apparently had some questionable data on trends in education. Inklaar says that the next version of the PWT, version 9, will fix the problems, and until it comes out, to use OECD data.

Well, I am mostly relieved. It's not really a methodology disagreement (except for the Barro-Lee education data). All of macro does not have to be scrapped, just yet. :-)

Thanks to Robert Inklaar for helping me out!

...But the growth economists I talked to about this mystery all expressed deep skepticism about these TFP data sets in general...

Growth vs. static efficiency

I have a new Bloomberg piece where I criticize John Cochrane, and conservatives in general by extension, for selling static efficiency policies as "growth" policies. The title is not the best (and the picture they use of John is also not the best; sorry about that). But the point is one I've really wanted to make for a long time:

Most of the so-called growth policies Cochrane and other conservatives propose don't really target growth at all, just short-term efficiency...Cochrane sells us on the need for growth policies by citing the undeniable benefits of long-term economic growth...But most of the policies Cochrane recommends are most certainly not things that would increase the growth rate for decades on end!...[S]uppose we cut taxes...the deadweight loss goes away...It provides a one-time bump, but nothing more...The same is true of most regulation.

You can read the whole thing here.

Now, Cochrane's piece is a very good one, as far as conservative policy manifestos go - it is non-polemical, thoughtful, and well-researched. It includes not just standard Republican planks like tax cuts, but also some things like increased spending on research. I think Cochrane would be a great chief economic advisor for the Rubio administration.

Nor is he trying to be dishonest here. Cochrane is a good guy. The focus on "growth", and the tendency to sell static-efficiency policies with paeans to the benefits of multi-decade compounding, is just a bad habit - a holdover from Reagan days. But nevertheless, I think it's sloppy. Policies to boost static efficiency should be able to stand on their own merits; they don't need to be oversold like this.

Robert Lucas in biology class

Back in August, a bunch of people were talking about Paul Romer and Bob Lucas and history of macro and stuff like that. Somehow I missed this post, where Brad DeLong dug up a Bob Lucas memoir and made fun of Lucas' college biology class exploits. For reference, here's a longer version of Lucas' story:

The only science course I took in college was Natural Sciences II - a biology course. We read a modern anatomy text, and also selections from Darwin, Mendel, and others... 

[T]here was nothing spooky about Mendel’s genetic theories. They were clear, they made some kind of sense (though there was nothing molecular in our Nat Sci II readings), you could work out predictions that would surprise you, and these predictions matched interesting facts. We did a classroom experiment with fruit flies, focused on eyes, and pooled the results. Our assignment was to write up the results in a lab report and compare them to predictions from a Mendelian model. I had not enjoyed the actual lab work but I liked writing the report and spent the better part of my weekend on it. It was the first time I can recall ever working out the predictions of a scientific theory from its basic principles and testing these predictions against experimental evidence. 

On Sunday evening, my friend Mike Schilder asked to copy [my report on the fruit fly experiment]. I agreed...Mike came back in half an hour, and told me: “This is a good report, but you forgot about crossing-over.” “Crossing over” was a term introduced to us to describe a discrepancy between Mendelian theory and certain observations. No doubt there is some underlying biology behind it, but for us it was presented as just a fudge-factor, a label for our ignorance. I was entranced with Mendel’s clean logic, and did not want to see it cluttered up with seemingly arbitrary fudge-factors. “Crossing over is b---s---,” I told Mike. In fact, though, there was a big discrepancy between the Mendelian prediction without crossing over and the proportions we observed in our classroom data, too big to pass over without comment. My report included a long section on experimental error, describing the chaotic scene that generated the data and arguing that errors could have been large enough to reconcile theory and fact. I handed it in as written. Mike, on the other hand, took my report as it stood, except that he replaced my experimental error section with a discussion of crossing over. His report came back with an A. Mine got a C-, with the instructor’s comment: “This is a good report, but you forgot about crossing-over.” 

I don’t think there is anyone who knows me or my work as a mature scientist who would not recognize me in this story. The construction of theoretical models is our way to bring order to the way we think about the world, but the process necessarily involves ignoring some evidence or alternative theories - setting them aside. That can be hard to do - facts are facts - and sometimes my unconscious mind carries out the abstraction for me: I simply fail to see some of the data or some alternative theory. This failing can be costly and embarrassing to me, but I don’t think it has any effect on the advance of knowledge. Others will see the blind spot, as Mike did with crossing-over, keep what is good and correct what is not.

DeLong makes fun of Lucas for rejecting chromosomal crossover. which is indeed a real thing, and the discovery of which won a Nobel in 1933. It does seem kind of lazy, actually. Even before Wikipedia, it wouldn't have been hard to go grab an advanced textbook and look up how chromosomal crossover works. Lucas is unhappy that it's presented as a fudge-factor, but by the time you're an undergrad you should be too old to depend on the teacher for 100% of your knowledge. If something isn't adequately explained to you, go look up how it works! 

Lucas says that this episode demonstrates a professional weakness of his - the tendency to want to over-simplify theory in order to "bring order" to the world. But I think it demonstrates something slightly different and more worrying: selective empiricism.

In his bio class, Lucas did an experiment on fruit fly inheritance. After the results didn't completely agree with the predictions of basic Mendelian theory, he attributed the discrepancies to experimental error - basically, to measurement noise. Fine (if slightly lazy). But then he takes the experimental result as support for the Mendelian theory, despite the presence of all that experimental error!

If the experimental situation was such a "chaotic scene," then any seeming agreement between Mendelian theory and the lab results might well have been an experimental error. So if college-age Lucas had really been an empiricist, he would have said "This experiment was such a chaotic scene that it provides only very weak support for Mendelian theory." Instead, he concludes that the experimental setup was reliable enough to support the theory that makes "some kind of sense" to him, but too unreliable to indicate the presence of additional phenomena like chromosomal crossover.

In other words, Lucas' conclusion from the experiment relied strongly on his own priors. Or if you prefer a frequentist term, he protected the null hypothesis. That has little to do with oversimplification; it's just a manifestation of confirmation bias. You pick the theories that make sense to you, and believe in them until the data decisively refute them.

But hey, who among us didn't have silly ideas in college?