This article is taken from the October 2023 issue of The Critic. To get the full magazine why not subscribe? Right now we’re offering five issues for just £10.

Writing intelligence history is problematic. The difficulties it presents are essentially twofold. First there is the issue of archives and records. As Calder Walton himself points out, there is today such a huge amount available, that to research the material thoroughly is very challenging.

However, some crucial areas remain secret, and in all likelihood they will never become accessible. For example, the archives of SIS — MI6, in popular parlance — are never released, and CIA’s releases are selective. There is thus a hinterland of records relating to “the epic intelligence war” that are not available to historians. This limits their perspective and may lend fragility and distortion to some of their analysis. However, this book’s extraordinary chapter notes do testify to the thoroughness of the research that has gone into it.

The second difficulty is how to rise above the sensationalism of the “spy story” and write something that delivers a serious historical perspective. I am immediately discouraged by the book’s title — presumably the publishers insisted on “Spies” in order to sell more copies. The intelligence war which this book maps in detail is about much more than spies. Thankfully, Walton, excellent scholar that he is, rises above the sensationalism.

The book is an outstanding work of history that contributes to our understanding of the fraught relations between Russia and the West over the past century. Particularly special is how the espionage anecdotes are fitted into the bigger historical picture. In this respect, the work is masterful: I cannot think of another academic account of Cold War espionage that does this so well.

Walton also understands the brutal truth about this conflict — that the intelligence was more important than the individuals who risked their lives (and often lost them, too) providing it. Today we incline to telling the human story, but this fine work of scholarship is really about the clashing interest of powerful nation states. Individual players are sometimes crushed by the weight of the events that their own risk-taking initiated.

Speaking from personal experience, those Western intelligence officials that were in the front line in this epic war paid little attention to the historical picture. The day-to-day professional pressures, the tactical aspects of high-risk intelligence gathering against such a formidable opponent as Soviet Russia were all-consuming.

The distinguished 20th century historian Tony Judt expressed the importance of being able to take historical distance in his essay “The Misjudgement of Paris”. “History is not written as it was experienced, nor should it be. The inhabitants of the past know better than we do what it was like to be there, but they were not well placed, most of them, to understand what was happening to them and why.” Though the insight is obvious, it is not often expressed.

We should admire the historians who are able to explain convincingly “what was happening … and why”. Walton is firmly of them, but the explanations are essentially historical. With events that are both very recent and current, they do sometimes seem to be a bit too tidy intellectually. The messiness of having experienced them cannot be entirely discounted.

Intelligence and Security Studies is a relatively new academic discipline, and it has been regarded as a sideshow to the mainline political, social, economic and environmental narratives of contemporary historical studies. The subject has also suffered from its close association with fiction. Until the Cold War was almost over, little authoritative material about espionage was available in the public domain.

The void was filled, in part, by fiction writers such as John Le Carré, whose best-selling novels were influential in creating a view of the world of espionage and, in his case, betrayal as well. It was taken for reality in the absence of alternatives. Walton’s work serves a wider purpose in asserting the claim of Intelligence and Security Studies to be regarded as an important aspect of the history of the 20th century, requiring the sort of detached and independent commentary that only a serious historian can provide. A number of leading university history faculties still remain to be persuaded, however. This book should help.

The great espionage cases that form the core of Walton’s account all share an important characteristic that is not much understood outside the profession. The case work, on the one hand, had to be very highly compartmentalised, known only to the smallest possible indoctrinated group; whilst the intelligence, in order to be relevant and useful, had to be used without prejudice to the source.

Resolving this tension, between fierce secrecy to protect a source’s identity and the use of intelligence in and by government, was a constant Cold War challenge. A prime example is the French Security Service’s recruitment in 1982 of the KGB scientific and technical line officer, known by the codename “Farewell”, just ahead of his leaving Paris to work in a key post in the Soviet Union’s military-industrial complex.

It deserves special mention because this was perhaps the most important espionage case of the later stages of the Cold War. The intelligence it produced was of such strategic insight that President Mitterrand quickly understood that it had to be shared with the United States. It was beyond the competence of any European government to digest or act on it.

The Soviet “intelligence requirements” documents that Farewell provided gave a precise picture of where Russian weapons technology lagged behind the West. They also gave confidence to the Reagan administration’s assessment that the Soviet Union could not match the US strategically without breaking its economy.

This revelatory article told the KGB that it had a major problem

Furthermore, the French Security Service were encouraged to publish in a quasi-official French journal an authoritative article on the VPK, the organisation which had allowed Farewell such authoritative and extensive intelligence access. Hitherto the West had not even known of its existence, let alone its importance. The appearance of this revelatory article told the KGB that it had a major problem, but by then the tragic Farewell was already in prison for the murder of his girlfriend. It did not take the KGB long to track the problem back to him.

Walton does acknowledge the importance of this case, but without the full contemporary context. This provides the best example of a source of prime importance being destroyed by the quality of the intelligence he produced. Once that intelligence had become the stuff of relations between nation states, as it did, Farewell was doomed as a spy — though strangely in this instance, he was already doomed as a murderer. His fate had already been sealed.

Another case of exceptional importance, occurring as the Cold War ended in 1991, was the defection of Mitrokhin. A KGB archivist, he transcribed and brought to the West a vast stash of the archives he had overseen. Walton acknowledges their importance in his note on methodology and sources, but he makes only a brief mention of the case in his text in relation to KGB activity in India.

I would have liked to see Mitrokhin’s role more thoroughly explained, because his information authoritatively answered so many questions about Russian Soviet intelligence activity and gave a wealth of new counterintelligence leads. No one had anticipated that a single source could provide such a sweeping and detailed view of KGB operation over a period of more than 50 years. It was an extraordinary conclusion to a long period of the potent Soviet espionage threat.

Walton’s masterly work suggests to me the publication of a second volume, covering the role played in the “epic war” by the other member countries of the Warsaw Pact and the West’s involvement against them. It is important to understand that the Polish, Czechoslovak, East German and Bulgarian services were all formidable, highly-resourced and effective extensions of the Soviet services. The Romanians and Hungarians were possibly less closely linked, but they were still willing to work to a Soviet agenda.

Central and Eastern Europe was also the cockpit where much of the activity described in this book took place, and the satellite services were almost as important as the KGB itself. Berlin, Vienna and Geneva were the sort of cities where, as one former Western intelligence officer described it, “you could actually hear the rustle of contact notes being written and filed”, as the intelligence services of East and West faced off against each other.

The book is thoroughly up to date, concluding with chapters on Putin and China. Walton’s conclusion about Russia is depressing — that Russia itself is the West’s problem more than Putin. He is probably right, given Russia’s inherent suspicion of the West and what successive Russian leaders perceive to be the West’s worst intentions towards them. However, I would argue that war in Ukraine is changing the fundamentals of this difficult relationship.

Russia without Ukraine will be a much diminished polity. The West with Ukraine as a close military and intelligence partner will have a strategic advantage in monitoring Russia’s future behaviour and attitude, rather as East Germany was able to penetrate most of West Germany’s state institutions.

A diminished Russia that has lost Ukraine to the West will remain a very volatile and dangerous neighbour, however, unless there were to be a revolutionary shift towards democracy in its domestic politics. That is unlikely at the moment but not to be excluded.

Walton’s chapter on the threat from China is titled “The New Cold War”. He explains that it is not like the Cold War at all. He is correct. We do need new terminology in order to avoid treating the PRC as if it were the Soviet Union. The economies of the developed West are intertwined with the PRC’s, whereas the Soviet/CMEA economies were in effect almost entirely separated. The geopolitical difference is striking.

Strategic calculations on either side of the relationship must take account this interdependence, which reaches deep into both economies and thus into the social and political order of both as well. The problem is more serious for China because its politics, such as they exist at all, are brittle. The mechanisms for handling social and political change are non-existent.

Following the massive diversion of espionage resources into counterterrorism, China is now the barometer by which the efficacy of the West’s national security policies will be measured, with all the complexities that new technologies, AI and data collection add to the various methods of intelligence-gathering. Walton suggests that these will amount to espionage climate change, making the stark assertion that “the age of the secret service is over” — but the intelligence war between East and West will continue.

Another large-scale change in the variable geometry of the secret services is happening, not that we can see much of it. I would nonetheless predict an enhancement of their role as we move into a world of much greater instability. Bellingcat is often quoted, by Walton as well, as evidence of the overwhelming importance today of open sources.

However, Bellingcat’s best intelligence insights have been achieved by the organisation gaining access on the Dark Web to closed databases, which they have been given or sold by disgruntled or mercenary human sources. Human spies remain alive, relevant and highly active. The secret services that run them are not going away — they simply need to become more multi-dimensional, especially with regard to the relevance of new technologies.

Those services will remain the ringmasters of one of the world’s oldest professions. A new age of secrecy is already launched, but the date stamps on the secrets may well, in the future, enjoy a shorter life.