The recent paper “20 Years of the Euro: Winners and Losers” by Gasparotti & Kullas estimates that the introduction of the euro increased the pro-capita income in Germany and the Netherlands, while it impoverished the other EMU members. Critics have poured on the statistical technique. In our opinion, a great deal of these criticisms also apply to a wide range of orthodox analyses.
Economists develop theories on how individuals behave in given socio-economic contexts, and debate on the effects of different rules of the game (different regulations, monetary frameworks, government interventions). Yet, with the partial exception of experimental economics and its laboratories, economics cannot compare “actual facts” with “counterfactuals”. People have preferences, expectations and knowledge of things present and past. As time goes by individuals and environments change for good. Hence, one cannot assess the consequences of a given policy by comparing individuals’ actions and reactions with what they would have done if they had been somebody else.
Yet, Gasparotti & Kullas used the Synthetic Control and devoted their energies to building a “counterfactual” situation. The technique goes as follows. For certain EMU members, the two researchers combined the historical income data of countries around the world that have not adopted the euro. By calculating the proper array of weights, they have built “synthetic” time series of pro-capita income that strictly fit the historical data of each EMU member within the 1980-1996 period. In other words, the authors have imagined that the performance of each euro country can be replicated by a mix of the performances registered for other non-euro countries. For example, Germany gets replicated by a combination of Bahrain (28,1%), Japan (35.8%), UK (26,4%) and Switzerland (9,7%); Italy by a combination of about UK (some 65%), Australia (about 30%), and residuals from Israel and Japan; France by Australia (50%) and the UK (50%). According to the authors, one can extrapolate the synthetic data from 1999. In their view, the data thus obtained are the “euro-counterfactual” situation, i.e. the performance of each EMU member if it had stayed out of the euro-area. The difference between the extrapolated data and the actual performance measures the gain/loss from having adopted the euro.
Euro-enthusiasts have raised various criticisms. The technique used in the article – they argue – is suitable for micro-analyses in which a small entity (e.g. a town) is compared with similar entities belonging to the same macro-environment (e.g. other cities located in the same territory or political unit). By contrast, dealing with different countries involves too much heterogeneity. One would hardly obtain meaningful results by comparing a town with a benchmark derived from other towns characterised by significantly different features.
Moreover, alternative exercises show that one can easily come to different conclusions when one builds the “counterfactual” by using a slightly different set of countries. For example, a synthetic Italy built by Switzerland, UAE, Brazil, Japan and Norway also looks as a good fit. However, under these circumstances Italy’s underperformance would start in 2007-2008 (the euro would play no role). Similarly, critics have shown that by removing Bahrain from the German “counterfactual”, Germany would become a net loser.
Of course, developing a benchmark for – say — Italy by imagining it is a combination of Japan, Israel, Australia and the UK is rather puzzling, possibly meaningless. Regardless of the quality of the statistical exercise, sound economic reasoning and common sense are the most appropriate way to evaluate any quantitative result. For example, even if one takes the synthetic counterfactual seriously, one must be ready to accept that the post-euro divergence might originate from causes other than the euro. In particular, the role of national policy-making is significant, and could have pushed the euro economies away from the path suggested by the statistics, despite the currency in circulation. From this viewpoint, therefore, the authors of the paper have not assessed the effects of the euro. Rather, they have analysed a set of country-specific features and policies, possibly interacting with the new currency. Clearly, specific conclusions about the euro are impossible.
In brief, nothing assures that economic structures and policies have operated in the same way across the various countries since the inception of the euro. In fact, country specific political features determine how an economy reacts to outside stimuli and shocks, regardless of euro membership. In other words, economic and political structures are not the same across countries, and do not remain stable over time. This is why the so-called “counterfactual” is meaningless. More generally, if we acknowledge that the notion of starting point is not limited to technical or scientific knowledge, but also includes the body of knowledge typical of each individual at given moments, all statistical exercises are more or less satisfactory descriptions of past quantitative patterns, but offer no explanations, let alone predictions. Aggregation makes no difference: even citizens in towns within the same country do not own or use the same knowledge, nor do they react to shocks in the same way.
Certainly, our doubts apply to the Synthetic Control method and, more generally, to all those attempts to use statistical analysis as a substitute for explanations based on logic and reasoning. Not only does number crunching provide deceptive narratives, It is also less than reliable per se: as we recently pointed out – see here on IREF and recently confirmed by Nature — about 50% of the statistical results in the social sciences cannot be replicated and must therefore be considered as fallacious.
