How did it come to pass that the progressive worldview that presented itself at the millennium — “like a land of dreams, various, beautiful and new” — mutated after 2001 into a darkling plain, where an exhausted West struggled to contend with ignorant armies clashing by night? This most pertinent question is posed by David Martin Jones in History’s Fools: The Pursuit of Idealism and the Revenge of Politics.

His book, scholarly, erudite and powerfully argued, provides reason enough to conclude that, unless policymakers and those who shape the climate in which policy is made acquire some inkling as to the answer to his question, further humiliation awaits.

Increasingly political parties have ceased to represent ordinary people

For it is clear as anything can be in politics that the dream of a conflict-free liberal order based on shared norms, open markets, open borders and an abstract commitment to social justice is dead and that any attempt to preserve its remains is bound to fail. He writes: “The truth is a morality and a political project in this form, whatever the quality of its ideals, brings nothing but distraction and moral and ultimately political instability.” As in the past, chagrin awaits those who believe that they have found a short-cut to heaven.

Jones is admirably clear about why the dream was doomed: the metropolitan elites which dominated government thinking in the US and Europe for 25 years in effect abandoned politics in the pursuit of utopian, transnational solutions, squandering opportunities for prudent and measured steps that might have strengthened the wealth and security of Western nations at a unique historical moment while, at the same time displaying a dangerously innocent belief that societies of which they knew remarkably little could be made to share their values and aims.

In doing so they took a long view of the future and a short one of the past. Now politics is taking its revenge. Jones quotes Hannah Arendt, who was no less scathing about an earlier generation of utopians who thought that their reading of history enabled them to mould the future: “There is some ludicrousness in the spectacle of these men . . . submitting from one day to another, humbly and without so much as a cry of outrage, to the call of historical necessity . . . they were fooled by history and they are the fools of history.”

China has been rewriting the rules of international trade while systematically constructing a sino-centric regional order

As the author shows, the problem is not simply that the governing elite failed to export their dreams to others, or that their idealism resulted in a degree of foreign policy incoherence that cost lives and greatly weakened Western influence and reputation. There were other undesirable consequences, including the promotion of their projects that hollowed out domestic politics, destroying the links between political parties and their traditional supporters. There is presently evidence of pushback, but increasingly political parties have ceased to represent ordinary people, becoming emissaries of centralised bureaucracies to whom they so unwisely ceded authority.

As Jones points out, politicians like Tony Blair, David Cameron, Theresa May, Emmanuel Macron and Barack Obama often liked to give the impression that they were above politics. As a consequence, there was a world of professional politicians and a world of citizens who did not share the views of the former and dangerously little interaction between the two.

In this project the state-funded Western academia acted as cheerleaders for “democracy and the global order”, losing their capacity for independent thought and intellectual integrity in the process.

Not the least service performed by Jones is his demonstration of the scale of the hubris and dogmatism displayed by those who saw their task as that of “embedding utopia” in a globalised world. At the millennium it was both academically and politically fashionable to assert as Anthony — now Lord — Giddens did that the developed nations of the world were “without enemies”. Globalisation and the interconnected market made war increasingly “unlikely”, Giddens wrote.

In the US Thomas Friedman asserted that “no two countries that have McDonald’s have ever fought a war against one another since they got McDonald’s”. Evidently Friedman does not drop in for a Big Mac in Moscow or Kiev on his European jaunts.

Jones’s book excels as an appeal for a return to a more sceptical and prudential kind of politics

“History is being driven in a coherent direction by rational desire and rational recognition,” Francis Fukuyama wrote in his famous essay in 1992. Although civilisations might clash, the appeal of democracy was such that its snowballing effect would prove irresistible. According to Fukuyama, “Mankind will come to seem like a long wagon train strung out along a road.” Some would be stuck in ruts or bivouacked in the desert while some would be attacked by Indians. Nevertheless, “the great majority of wagons will be making the journey into town and most will eventually arrive there.” In disparaging such views Jones quotes Michael Oakeshott: “A single homogenous line of development is to be found in history only if history is made a dummy upon which to practise the skill of a ventriloquist.”

When it was pointed out to the end-of-history progressives that China showed no sign of heading in the direction of the democratic destination ordained for it, let alone arriving there, becoming under Xi Jinping both more repressive and more aggressive, it was confidently asserted that the golden straitjacket which China’s embrace of capitalism necessarily entailed would inevitably bring political liberalisation in its wake.

According to Friedman, in the high-tech age either China wouldn’t get richer or it wouldn’t be so authoritarian “because what the Chinese government can get away with now is very different from what it will be able to get away with once it is fully integrated into the Electronic Herd”.

The reality is that China has been busy rewriting the rules of international trade while systematically constructing a sino-centric regional order. As Jones reminds us, China’s belt and road policy and the construction of a China-Asean free trade area will give substance to China’s hegemonic ambitions. This is not quite how the architects of the New World order said things would turn out.

Jones’s writing on politics and international affairs lies in the tradition of conservative scholars such as Oakeshott and Kenneth Minogue, whose profound scepticism about rationalist projects for the improvement of mankind has plainly been a major influence on him. “Such projects assume that they may avoid the difficulties of life by engaging in a scheme in which the ends have been determined for them,” he writes. “They explicitly pursue a state of perfection, all the while neglecting the joys and sorrows of our present temporality.”

His book excels both as an appeal for a return to a more sceptical and prudential kind of politics, and as an authoritative and penetrating analysis of the West’s present predicament and the steps by which it arrived there. It is perhaps not quite as pessimistic as the previous paragraphs suggest. He acknowledges that we have been here before: the triumph of the West has often been accompanied with, or closely followed by, predictions of its impending doom. Moreover, democracies have in the past found a way of somehow stumbling through.

His conclusion offers a degree of hope to all those, including the present reviewer, who are inclined to the view that for at least the time being the nation state provides the only possible foundation for representative government: “It is perhaps this capacity [the ability to somehow stumble through], along with the return to a more sceptical politics of balance, practised by free citizens in their national states, that may still give them an edge over their politically religious progressive transnational and illiberal authoritarian rivals.” Our political destiny depends upon it, but it looks like being a frighteningly close-run thing.