Last week, Nick Cave released his latest album, Idiot Prayer. Recorded live during lockdown from Alexandra Palace, it is a sombre and stately collection of his songs, played solo on piano and sung by the now-63-year-old musician with both conviction and passion.

Such stripped-back LPs have long been associated with the older artist trying to imbue their oeuvre with sincerity and gravitas, and are often interpreted by the unadventurous as an opportunity to slather their back catalogue in lavish string arrangements, leading to the incongruous spectacle of the likes of Rod Stewart performing “Sailing” with the Royal Philharmonic Orchestra. These albums are usually made with a mixture of big budgets and not a little rock star ego. And most of the time they sell in decent quantities and are then forgotten when the next album “proper” is released.

One could almost call him middle of the road, if this particular route was the one to hell

Idiot Prayer is a rather different beast. It was filmed by the well-respected cinematographer Robbie Ryan and released cinematically as a live stream in July and in longer form in early November. Its appeal, as with the other documentary films in which Cave has appeared in, 2014’s 20,000 Days on Earth and 2016’s One More Time With Feeling, is that the artist himself is a strange mixture of the entirely accessible and the remote and enigmatic.

When he sings “Black Hair”, taken from his masterly album The Boatman’s Call, he has been entirely candid about its inspiration being his relationship with PJ Harvey and its subsequent, unhappy end, meaning that the lyrical archaeologists who enjoy delving through the canons of great songwriters and arguing about this or that interpretation will be put out to have their hunt thus frustrated.

Yet as Cave returns to many of the mournfully beautiful piano ballads with which he is most closely associated – “Into My Arms”, “The Ship Song” and “Far From Me”, to name but three – he seems to be embracing his middle-aged destiny as a dark-haired, stick-thin prophet of love and loss. One could almost call him middle of the road, if this particular route was the one to hell, paved with good intentions or not.

This is a considerable shift from the beginning of his career as the hard-living frontman of “the most violent live band in the world”, The Birthday Party. The young Cave, the son of an English teacher and a librarian, grew up with both an abiding love of literature and an interest in rare and exotic pharmaceuticals, both of which would swiftly become the defining features of his life and career. He was, by his own admission, addicted to heroin and other drugs for many years, and last year briskly listed all of the various drawbacks that come with such an addiction:

Bashed up in police stations, dehumanised in rehabs, near-death experiences, suicidal thoughts, routine overdoses, reduced motivation, broken bones, being ripped off, liking Charles Bukowski, social and physical anhedonia, herd mentality, dead friends, fucked up relationships, abscesses, car accidents, psychosis, reading The Hobbit, malnutrition, creative impotence, epic time-wasting, singing flat (still working on that), talking shit (still working on that too), life-threatening diseases, and not ringing my mother on her birthday.

He also conceded that “I’m not sure if I have quite reached the same highs as those times while sober”, even as he noted “I also haven’t reached the kind of lows that eventually came with taking those lovely substances”. Some of the albums that he recorded and released while he was still using drugs are now regarded as masterpieces, not least 1997’s The Boatman’s Call, a sombre, piano-led collection of beautiful songs that can now be seen, for better or worse, as a template for the styles and approach that he has pursued in the second half of his career. Yet the music that he has made over the past two decades after foreswearing drugs and alcohol is often phenomenal, whether it’s his 2004 double album Abattoir Blues/The Lyre of Orpheus, 2016’s bracing Skeleton Tree – released in the aftermath of his son Arthur’s accidental death in 2015 – and last year’s deeply sombre, brooding Ghosteen.

As a musician, Cave is strongly in the vein of such poetic singer-songwriters as Bob Dylan and Leonard Cohen, although he presents himself as a more energetic, direct figure, unafraid of seeming or looking ridiculous.

Popbitch frequently has a section in their newsletter devoted to Cave’s more unlikely appearances, which have included everything from being seen at a pantomime in Brighton enthusiastically waving a giant foam hand and singing along to “It’s Chico Time” to his being a keen paintballer in Surrey. I remember seeing him once at Victoria station, scurrying for a train and being pursued by an over-enthusiastic admirer asking for a selfie; the expression on his face, mingling exaggerated panic with amusement at his predicament, was something to behold.

Cave has never discussed his personal politics, rejecting any attempt to label him

Yet the past few years have seen Cave expand his horizons from releasing albums and conventional live shows in favour of something more intimate. He has regularly been answering questions from his admirers on his website, in a section known as the “Red Right Hand files”, after a song from his Murder Ballads album. He has answered 125 of these questions to date, and has done so candidly, humorously and provocatively, sometimes all at once. These have ranged from the deeply personal, often as he talks about his emotions engendered by the death of his son, to the more frivolous.

After he remarked that the piano that he performed on in the recording of Idiot Prayer was a Fazioli, and that it was “the most beautiful instrument that I have ever played”, he wrote a humorous sketch about his trying, and failing, to use his standing as a world-famous rock star to be sent a free piano by the manufacturers. His more over-enthusiastic aficionados took this seriously and so Cave was driven to ask his more vocal fans to stop bothering the “wonderful people” at the company with demands that he should be gifted one of their pianos.

As the old structure of musicians making money from a mixture of album sales and live performance disappears, so many have re-examined the relationship between artist and audience. In practice, this has meant things like expensive meet-and-greets before shows, when fans can pay hundreds of pounds to have your photograph taken with their favourite star, and their broad grin of excitement will probably be equalled by the artist’s slightly pained expression of contractual obligation.

Cave has been attempting something altogether different, not only in his “Red Right Hand files”, but also in a series of solo live performances where he answers unfiltered questions from the audience candidly and openly. In practice, this has meant that many of his admirers have used the opportunity to declare undying love for him and his works, which he has responded to graciously if wryly, but others have gone to his shows with other intent in mind. The best of all of these came at a show in Brighton, when one wag put up his hand and said “Nick, I live in your old flat in Hove. Could you tell me where the stopcock is?”

Cave remains an enigmatic and unknowable figure

When he is not being asked about plumbing, Cave’s opinions on politics and culture wars are just as keenly sought. As befits a man of his thoughtfulness and nuance, he has refused to take the kneejerk left-wing line of many of his peers. He played in Israel despite Brian Eno loudly calling for a boycott and justified his decision by calling such a snub “cowardly and shameful”. He described both the behaviour of the Israeli government and Hamas as “grotesque”, saying that he wished for the conflict to be “ended via a comprehensive and just solution, one that involves enormous political will on both sides”. Such a measured and balanced approach has led to criticism of his views, but he has responded robustly to his opponents attempting to “call him out”. He said last year that he was made uncomfortable by “any system of thought, including ‘woke’ culture, that finds its energy in self-righteous belief and the suppression of contrary systems of thought.”

He went on to say that “Antifa and the Far Right, for example, with their routine street fights, role-playing and dress-ups are participants in a weirdly erotic, violent and mutually self-sustaining marriage, propped up entirely by the blind, inflexible convictions of each other’s belief systems. It is good for nothing, except inflaming their own self-righteousness”, before concluding that, “Wokeness, for all its virtues, is an ideology immune to the slightest suggestion that in a generation’s time their implacable beliefs will appear as outmoded and fallacious as those of their own former generation” and “this may well be the engine of progress, but history has a habit of embarrassing our treasured beliefs”.

There are many leading figures who have set themselves against woke culture, but most of them, including the likes of Piers Morgan and John Cleese, have never been taken particularly seriously by those who they rail against. Cave is an altogether different proposition. He has never discussed his personal politics, rejecting any attempt to label him as liberal, conservative or anything else, but for a man possessed of an enquiring and open mind. It is also heartening to find that, just as he has offered succour and comfort to many who have lost someone dear to them, he is also prepared to point out that the emperor has no clothes and that ideologies that many have unquestioningly signed up to in their entirety should be re-examined, questioned and rejected, if needs be.

Cave remains an enigmatic and unknowable figure, despite this apparent candour and now newfound willingness to engage with his audience without the moderating influence of publicists or managers. For one tribe or another to attempt to “claim” him as one of their own would be a mistake, because he is, by his own admission, a contradictory and complex man, whose statements should not be taken as gospel.

And yet it is this willingness to admit his flawed humanity that is of a piece with his complex attitude towards God and faith. As he wrote in December 2018, “Do I believe in God? Well, I act like I do, for my own greater good. Does God exist? Maybe, I don’t know. Right now, God is a work in progress.” Cave himself, in all of his appearances and statements, asks that he, too, should be seen as a work in progress rather than the finished article. Judging by the affection and adulation that he receives; many people are only too keen to join him on this most uncertain of journeys. We can only hope that the destination ends up being a fulfilling one.