Today, Tate Britain’s Sidney Smith designed 1893 portico frontage is draped in the brash neon of Chila Burman’s reflections on a childhood in Blackpool. A glowing Potemkin family ice-cream van perches incongruously on the gallery’s steps, rendered in technicolour.

Why graffiti a much-loved institution?

And above? The, oh so predictable and self-loathing, transformation of the statue of Britannia, at the apex of the 1890s portico, into the goddess Kali, the destroyer. The Tate’s own puff describes K

ali as, “the Hindu goddess of liberation and power”, whereas the Encyclopedia Britannica has a slightly different take, “goddess of time, doomsday, and death, or the black goddess.” Her name translates from the Sanskrit as, “She who is Black” or “She who is death”.

The clunking analogy made by this is, no doubt, considered daring by those who commissioned it, including Roland Rudd, Remainer-in-chief. However, it is trite, obvious, and overwrought, by the rest of us, who see the building as a repository of the creativity of British artists over the centuries. It is not another occasion for cultural cringe and shame. Yes, okay, fine, but why do we have to pay for it? Why graffiti a much-loved institution?

Behind the gallery, in the slightly mangled grid-plan of the late-nineteenth century streets, is where the London County Council built its first estate for the working poor. Turner House, on Herrick Street, delights in Queen Anne style and Fabian socialist Arts and Craft detailing. Vincent Street, at a slight angle to true, has Lutyens’s 1920s foray into social housing, a chequerboard estate built by the Duke of Westminster as “housing for the working class”.

These became a cause célèbre in 1990 when Westminster Council, then led by the Tesco heiress, Dame Shirley Porter, attempted to overturn the 999-year lease, granted by the Grosvenor estates. The case was known as Westminster vs Westminster and ended, not surprisingly, in victory for Westminster (Duke of)

. The High Court decided that the working class had not been abolished either by a court ruling in 1955, or by the Thatcher premiership. Joe Ashton, the firebrand socialist MP for Bassetlaw, put it pithily: “The Lady Porters will always need the working classes to clean up after them. It shows that nobs like the Duke know more about how the world ticks, than the heiress to a Tesco fortune.”

Today, working peers and MPs, mix with recent migrants, journalists, and young professionals in the stripped-down, Georgian style flats.

Look closely in gardens, in the back ways into tenements, and the umbra of new-build hotels in the area, one can just see the floor plan of something once visionary, but what later became a by-word for corruption and dereliction.

Why does Millbank Tower squeeze uncomfortably into a truncated lozenge? What’s with the pointless, tarmacked ditch, alongside Cureton Street, now filled with pot plants and washing lines? Why is there such obvious, but seemingly planned, subsidence in the car park on John Islip Street? Why does Marsham Street take such an odd angle, like a broken arm? And who are the supposed ghosts in the cellars of the Morpeth Arms, built in 1845, where Millbank juts away from the river?

All these discrepancies can be traced back to the vision of Jeremy Bentham, the incompetence of

the first architects, Thomas Hardwick and John Harvey, and the genius of the builder of the British Museum, Robert Smirke. As you wander around, poking into back yards, you are on the site of the world’s first scientifically designed penitentiary.

Envisaged by Bentham as a (for its time) humane panopticon prison, where all could be seen, toiling, and praying for their own betterment, by a central scrutiniser.

Bentham bought the land which was then a marsh where duck were still being shot by the owner, the Marquess of Salisbury. Bentham did this with £12,000 on behalf of the Crown. Though his vision was followed to an extent, the work went to Harvey, but the foundations failed and, when Smirke was brought in, windows were popping spontaneously, and walls were crumbling.

Smirke was an innovator. Realising the failings of the underpinning, he created the first concrete raft foundations in these islands since the departure of the Romans, 1,400 years earlier. Archaeologists recently discovered his foundations to be robust and wildly over-engineered.

With more than a little design nod to the Bastille, Smirke’s “gothick” inspired edifice was built as a national prison, with one inmate to a cell. But as is the way of these things, good intentions go awry. It became fetid an

d crowded.

Closing as a normal prison, it became instead the embarking place for transportation, with upwards of 6,000 souls a year passing through its gates to Australia, Norfolk Island and Bermuda. These included Fenians, Chartists, poets, and prostitutes. The Tolpuddle Martyrs were amongst the cargo, along with the more traditional roster of thieves, forgers, footpads and felons. The prison was built in a hexagon, and today that hexagon can still be traced, softly, even in Google maps. Slicing between the Tate and Millbank Tower is even a surviving part of the outer ditch wall. Today instead of prisoners, the estates built on its remains house amongst ordinary citizens, apartments of parliamentarians.

And what of the ghosts of the Morpeth? It was built to refresh the wardens of the prison, and supposedly in its cellars are the shades of the pre-embarkment convicts. But as I walk past it on my nightly perambulations, it is not prisoners I hear clanking their irons, but a different measure of ghostly clink, that of glasses held aloft and the muffled hubbub of an ordinary Thursday night at the pub.