The room where I write is busy. The walls are covered in portraits, of prime ministers Rosebery and Salisbury, and newspaper proprietor Lord Northcliffe. The Empress Elisabeth of Austria is up there too and the gung-ho 19th Baron Willoughby de Broke. This isn’t an art gallery but my flat in south London, and the portraits are caricatures published in the late-Victorian society magazine Vanity Fair.

It has been four years since I bought my first caricature — of Telegraph owner Edward Levy-Lawson. Now, my partner and I have amassed a collection that totally outstrips the space available. A soldier, he likes military men and explorers — “people that actually did something” — and thinks most of my choices frivolous since they reflect my work as a writer specialising in the aristocracy and country houses. It is, I admit, a bizarre preoccupation, collecting caricatures of Victorian men in suits. But each tells its own story.

There have been several magazines called “Vanity Fair”, but this one was founded in 1868 by Thomas Gibson Bowles. Vanity Fair was Bowles’s baby, written for those “in the know”, as Roy T. Matthews and Peter Mellini put it in their 1982 book In Vanity Fair. “Members of the Smart Set delighted in finding themselves caricatured in prose and picture. For them, Vanity Fair summarised each week the important events of their world.” The magazine was self-consciously conservative, “supportive of the Conservatives and Disraeli, the Church of England, imperialism, the Crown and the class system. Gladstone and the Liberals were the sworn enemies [and] the Russians untrustworthy”.

While the magazine carried word games, book and theatre reviews, and political news, it is the caricatures for which it is best known today. The first of these appeared on 30 January 1869, of Benjamin Disraeli, with the caption “He educated the Tories”. It was drawn by the Italian artist Carlo Pellegrini, who initially took up the pseudonym Singe. Later, this changed to Ape. In 1873, Bowles hired Leslie Ward, who drew as Spy.

Subjects for caricature varied, and include rowers, freemasons, boxers, bankers, politicians, explorers, inventors, yachtsmen, and actors. A haunting, posthumous image of Robert Falcon Scott appeared in 1913. Others made multiple appearances: Winston Churchill in 1900 and 1911, and his father, Lord Randolph Churchill, twice in 1880 and again in 1889.

Of the 2,300 caricatures that appeared, fewer than 20 were women. This, says John Wilkes, a longtime dealer of Vanity Fair prints, was because a “gentleman would not portray a woman in a bad light, so they would have to do it more as a portrait than as a caricature”. But the chosen women packed a punch: Queen Victoria appeared in 1897, and Mrs George Cornwallis-West — the former Lady Randolph Churchill, Winston Churchill’s mother — in 1912. My favourite is the captivating Empress Elisabeth of Austria who hangs on my wall; “Sisi”, who was mad about hunting, was assassinated in 1898.

Not all were great works of art, however. The novelist Anthony Trollope featured in Vanity Fair in 1873, and despised his Spy portrayal, being especially incensed about the depiction of his admittedly bulbous right thumb. Later, Trollope’s biographer James Pope Hennessy described Trollope as looking like “an affronted Santa Claus”.

A description accompanied each caricature, along with a caption; former chancellor Lord Halifax has one of the best — “he fell off his horse into the peerage”. Bowles used the word “caricature” rather than “cartoon”, for the images were not supposed to be realistic, but an exaggeration of the individual. In 1872, he wrote that the purpose of the drawings was to portray the victims “not as they would be but as they are”.

But who would be the men and women of today? Newspapers editor — whatever one thinks of their influence — would certainly qualify. Geordie Greig of the Daily Mail, John Witherow of The Times, and Kath Viner of The Guardian. The newly minted editor of the The Sunday Times, Emma Tucker, too, and the shy and rarely party-going former editor of the Mail, Paul Dacre.

And then the hacks and broadcasters: Sir David Attenborough, in the jungle; Sir Trevor McDonald and Emily Maitlis in the studio, and Giles Coren at home, as well as recent Orwell Prize-winner Janice Turner. Owen Jones, too, to coincide with his 2011 book Chavs, which, like it or not, caused a sensation. And not forgetting the late Marie Colvin, the Sunday Times war correspondent who was murdered in Syria in 2012.

Outside the media, the list mushrooms: authors, actors, sportspeople, soldiers, educators, entrepreneurs, and social media stars. Teachers too: Anthony Seldon, former headmaster of Wellington College, and Katharine Birbalsingh, the sometimes controversial founder of the free school Michaela Community School. I cast a vote for Harry Dalmeny, the fabulously well-connected UK chairman of Sotheby’s, who once held an auction dressed in a suit cut away at the back to reveal suspenders.

Later editions of Vanity Fair seemed preoccupied with Oxford and Cambridge rowers, and today’s sporting crowd would include Andy Murray and Sir Mo Farah, alongside a gaggle of Olympians, probably including Zara Tindall, the Eventing daughter of the Princess Royal. Businessmen were always welcome in Vanity Fair; in 1911 Harry Gordon Selfridge, founder of the eponymous department store, appeared in the magazine. Today, I’d nominate James Dyson and Karren Brady. Campaigners, too: Doreen Lawrence would be a worthy figure, and Nimco Ali.

And then to world leaders. Vanity Fair ran an entire series on monarchs: Alexander II of Russia in 1869, King Humbert of Italy in 1878, and Alphonso VIII of Spain in 1893, amongst others. Today, Vladimir Putin and Xi Jinping would make the cut, alongside Benjamin Netanyahu, and Jacinda Ardern of New Zealand. One could suggest an EU series: Emmanuel Macron, Angela Merkel, and Michel Barnier. Alongside the politicians are the international super-rich — billionaires Bernard Arnault, George Soros, and Bill Gates — and the tech giants: Mark Zuckerberg of Facebook, and Jack Dorsey of Twitter.

In 1889 Bowles sold Vanity Fair to Arthur H. Evans, but before that managed to squeeze in a healthy number of now-apparently ordinary aristocrats — for that was the way of his world. My collection includes a wonderful caricature of the 5th Duke of Buccleuch from 25 January 1873, who stands, hat under his arm, holding a copy of the Daily Telegraph. The current duke, the 10th, a quiet man, was when I interviewed him in 2017 the largest private landowner in Britain. He too would make today’s list. As would all of the key members of today’s royal family — even, and especially given recent news, the Duke of York. It is a pity that no magazine today continues Bowles’ legacy — though undoubtedly lawyers would strip the fun out.

 The Liberal politician Joseph Chamberlain was the last caricature to appear in Vanity Fair, on 14 January 1914. Soon after, the magazine was absorbed into “Hearth and Home” magazine. Within six months, the First World War had begun, and Chamberlain was dead. “The way of life Vanity Fair had chronicled and caricatured would begin its agonising death in the trenches of France,” write Matthews and Mellini. The party was over. Today, the remnants can be found in musty antiques shops down side streets, or on eBay. Back in my flat, we relive the party most days — with Lord Salisbury surveying the scene.