You may have seen lots in the media about “doppelgangers” recently. Not the folk who look alike — which apparently is far more common than most realise, but the economic kind. Much of the analysis that purports to demonstrate a negative economic effect from Brexit is in fact based upon doppelganger models. But what are they? And why do they work so well in theory but fail in practice? 

Doppelganger models — or the Synthetic Control Method to give them their proper name — are a method to try to test counterfactuals. What ifs. Like Philip K Dick’s book The Man in the High Castle, which considers what might have happened to the world if the Axis Powers had won World War II, or Apple’s TV series For All Mankind, which considers what might have happened to the world if the Soviet Union had been the first to put a man on the moon. Brexit doppelgangers are an attempt to answer what might have happened to the UK economy if the UK had not voted to leave the European Union on the 23 June 2016.

These models are constructed by comparing the actual historic UK GDP performance over a set period of time and then comparing that to the actual historic GDP performance from a blended range of other countries over that same time period. The modeller selects his comparator countries and then changes the proportions of each in his “doppelganger” until it produces a GDP performance which broadly matches the actual historic economic performance of the UK over that same time period. 

The modeller then rolls forward in time to see how that doppelganger performed after Brexit — comparing that to the actual performance of the UK economy. The difference between the two is the supposed loss in GDP caused by Brexit.

For instance John Springford’s doppelganger model (the Centre for European Reform) starts in Q1 2009 and is made up of 22 of the 36 OECD countries, with the US being the largest contributor at just over 30 per cent, Germany coming in at about 15 per cent and New Zealand boasting about 14 per cent (Austria, Belgium and Japan are included in the model but zero weighted). Springford’s doppelganger (which is publicly available) reports a 5.5 per cent higher GDP growth than the actual UK GDP growth over the period June 2016 to June 2022.

The US Investment Bank Goldman Sachs recently published its own analysis of the economic cost of Brexit based on its own doppelganger model (for which no details on country constituents, weightings or time periods are provided). It reports a Brexit effect of 5 per cent. But interestingly the Goldman Sachs report states that this 5 per cent is their core scenario because “we also find that the magnitude is sensitive to the exact specification of the analysis, with a range of 4—8 per cent”.

This statement gives a clue about the first “problem” with doppelganger models. We don’t know exactly what Goldman Sachs meant by “sensitive to the exact specification of the analysis” (presumably the countries included and their weightings), but we do know that these changes had a significant effect on the modelled outcome. In football terms, the difference between taking a penalty and hitting the corner flag to the left and then taking another and hitting the corner flag to the right.

You see there are a large number of countries and country weightings that one can model which would produce a close match to historic UK GDP growth over a set time period. But each selection, once extrapolated forward, will produce hugely divergent outcomes. So on what basis do you choose which countries to include and which weights to apply?

As Graham Gudgin and Saite Lu wrote in their paper “The CER Doppelganger index does not provide a credible measure of the impact of Brexit”:

The CER doppelganger index for GDP…includes very different economies like Greece which had a totally distinctive experience of economic growth in the 2009-16 period, and countries like Iceland where the economic structure is unlike that of the UK, or indeed of anywhere else. It also includes Ireland where the GDP data is highly distorted by the huge inflow of profits from multinational companies attracted by Ireland’s tax haven policies. Large weights are given to continental economies with large raw material exports, including the USA, Canada and Australia.

Yes, you can create lots of different doppelganger models with a mixture of countries which grew higher, lower and about the same as the UK between 1Q 2009 and 2Q 2016 but which together (at different weightings) broadly matched UK historic growth. But that doesn’t mean that your doppelganger would look anything like the UK during any other time period — in the past or the future.

Because it isn’t just the choice of countries and their relative weightings which matters but also the time period chosen. The CER doppelganger model begins in 1Q 2009. However, if the model had begun instead in 1Q 2008 — when the divergence between the UK and US economies began as the graph below illustrates — not only would the original doppelganger model not match the historic UK GDP performance, but the post Brexit outcome it would “show” would also be vastly different (I know because I did so here).   

Which brings us to the next big problem with doppelganger models. Let’s assume that by some miracle you have identified a basket of countries which somehow, when combined in precisely the weights you have chosen, exactly replicates the UK economy. The same balance of services to manufacturing and agriculture. The same balance of consumption, investment, exports and imports. The same demographic profile. The same immigration rates. The same balance of payments, debt to equity, tax rates, energy use/price etc. The one thing you can 100 per cent guarantee is that the further you move forward into the future the less alike your doppelganger and the real UK would become. Because all of the above — and much, much more — would diverge over time. 

As Catherine McBride explains it in her piece “Goldman Sachs: Wrong on Brexit”:

A doppelganger is technically a twin. The nearest geographical examples of ‘twin’ countries are the result of political divisions of an island or peninsula — such as Haiti and the Dominican Republic, or North and South Korea. Both are great examples of why doppelganger economics doesn’t work. Haiti has a GDP per capita (PPP) of about $3,000 and a population of 11.7 million, while on the other side of the island, the Dominican Republic has a GDP per capita (PPP) of around $20,000 and a population of 10.5 million. Similarly, North Korea’s GDP per capita (PPP) is estimated to be under $2,000, while its near ‘twin’ South Korea’s GDP per capita is around $50,000. Both … are good examples of how countries with similar locations, land masses and climates can have hugely different economic outcomes due to contrasting government policies or cultures.

Okay, I know this is an exaggerated comparison. I am not trying to claim that the UK economy has diverged to the extent of South and North Korea. But in the real world, since 2016, haven’t we experienced some pretty rare and seismic economic events? There was Covid, for example, with economies shut down completely and supply chains collapsing. There was the Russian invasion of Ukraine, and its effects on prices and energy supply. 

And haven’t different countries made vastly different decisions on how to react to these events? Whether it be to lockdown or not lockdown (and if to lock down how, when and for how long). Whether it be whether, how and how much to engage in fiscal stimulus? Whether to raise or cut taxes. Haven’t different countries experienced different levels of supply chain disruption depending on their geographical position and trade patterns? Hasn’t it made a huge difference whether countries were energy sufficient, relied on imports or were net energy exporters? And even their policies on whether to shift this explosion in energy costs onto consumers, businesses or the Government (taxpayers)?  

In normal times we would expect the UK economy to naturally diverge from any doppelganger. But considering the period 2016-2022 included more than one “once in a generation” event, I think it’s reasonable to assume that inter country divergence (and not just for the UK) is likely and inevitably going to have been considerably greater than “normal”.

To illustrate the point I wish to look at just one example: the USA. As Graham Gudgin outlined in a presentation at the UK In A Changing Europe, whilst the USA only represents about 30 per cent of the CER doppelganger, the economic performance of the entire doppelganger is almost identical to that of the USA alone between Q1 2009 and Q2 2022 (the time period covered by the CER doppelganger):

What this effectively means is that when the CER produces the chart below and states that the cost of Brexit has been 5.5 per cent, they are saying that any and all divergence between the UK and US economies after the vote to leave the EU was caused by Brexit. Nothing. Not a penny or a cent is ascribed to any other event at all. Not COVID, not the energy crisis, not the different decisions made as a result. None of them mattered one iota. Brexit was and is all. 

Is this realistic? Well let’s look at the data. First the post-COVID fiscal stimulus.

In the USA, according to Ernst and Young, Trump’s $2.2 trillion 2020 CARES Act increased US gross domestic product (GDP) by $1.7 trillion (9 per cent of GDP) over the next 12 months alone. And just a year later the Biden administration announced a further $1.9 trillion stimulus package which the OECD forecast would add as much as 3 per cent to US (and 1 per cent to global) GDP. 

The Brookings Institution was even more bullish, arguing that by their calculations the package would boost USA economic activity, as measured by the level of real gross domestic product (GDP), by about 4 percent at the end of 2021 and 2 percent at the end of 2022.

In total the US post-COVID fiscal stimulus exceeded $5 trillion dollars versus 2020 GDP of just over $21 trillion. That’s a fiscal stimulus approaching 25 per cent of GDP.

To put this into perspective, UK GDP is just over $3 trillion dollars and current estimates of the total cost of UK government Covid-19 measures range from about £310b ($393b) to £410b ($520b). In total official figures show that spending in 2020/21 was about £179 billion ($226b) higher than had been planned before the pandemic for that year. 

So the UK fiscal stimulus (before any discussion over the relative quality) was at most less than half that of the USA. As the Financial Times reported in March 2021, in comparison to the almost non-existent fiscal intervention in Europe the “US stimulus package leaves Europe standing in the dust.” 

Is it possible that at least a proportion of the 5.5 per cent higher GDP growth achieved by the USA versus the UK since the Brexit referendum might have been a result of a US fiscal stimulus roughly double the amount of the UK rather than because of new non-tariff trade barriers between the UK and the EU?

And what about energy. 

Whilst the US has moved from being the largest importer of oil and gas in the world to a net exporter in the last decade, the UK has been moving in the opposite direction.  As the chart below shows, US gas exports have been growing in a linear direction since 2014 and in 2016 (the year of the Brexit referendum) exceeded imports for the first time.

Meanwhile the UK has been experiencing the exact opposite process.

And as you would expect, these shifts (among other things) have had a significant impact on domestic energy prices for both consumers and businesses. As the chart below shows, UK and USA industrial energy prices started to diverge in 2003 but especially from 2016 (the year of the Brexit referendum). 

The pattern is broadly similar for domestic electricity prices.

Indeed, according to Offshore Energies UK (OEUK), the spike in oil and gas prices following the Russian invasion of Ukraine caused the UK energy import bill to more than double in 2022 to £117b. Equivalent to £4,200 per UK household. And in case you were wondering, no the USA did not experience the same issue.

    

Nobody is denying that the UK economy has underperformed compared to the US since 2016. But as the chart below demonstrates, so has all of Europe and the UK economy has broadly matched the rest of Europe over that time period.

Is it possible that this is because the UK has dealt with COVID in a manner more similar to the rest of Europe than the USA? Provided post-COVID stimulus in a manner and scale more similar to the rest of Europe than the USA? Been reliant on gas imports and endured higher energy and electricity costs more similar to the rest of Europe than the USA? 

Or is it more likely to be because of new non-tariff trade barriers between the UK and the EU?

Doppelganger models have their place, but they work better in theory than practice. In practice you need to back test/sanity test the modelled outcomes to see if they make sense in the real world. As Simon French, the Chief Economist at investment bank Panmure Gordon commented, the problem with the Goldman Sachs doppelganger model — or the Centre for European Reform’s model — is that they are telling us if the UK had not left the EU then the UK would have grown GDP in line with the USA “which has had huge fiscal stimulus, balanced energy market and less stringent CV-19 restrictions over the period”.

As I hope I have made clear above, that is simply not plausible in the real world.

Or to look at it another way, below is a chart of cumulative growth in real GDP in Europe since Q2 2016 (courtesy of economist Julian Jessop):

For Goldman Sachs’s doppelganger to be correct — that Brexit has indeed reduced UK GDP by 5 per cent — that would mean Goldman Sachs are arguing that (inside the EU) the UK would have grown GDP by 13.4 per cent over a period the Eurozone only grew 8.6 per cent and Germany 5.8 per cent.

That is 55 per cent higher GDP growth than the Eurozone and 131 per cent higher GDP growth than Germany. For the Centre for European Reform’s model to be correct, the UK would have had to have grown nearly two and a half times faster than Germany.

Again, it should be quite clear to everybody that this is simply not plausible in the real world.

The synthetic control method (Doppelganger models) have their place. Studying “what ifs” is a fantastic theoretical exercise. A way of testing assumptions and sensitivities in models. Whether that be in science, social science or economics. But ever since these models were created users have been warned about their limitations and their flaws and the need to sanity test the outcomes.

These Brexit doppelgangers do neither. I am as fond of “what ifs” as the next man. But if you like your counterfactuals to be plausibly believable, you would be better off reading The Man in the High Castle than these. And The Man in the High Castle contains characters who can travel between parallel universes…