They say everyone who’s old enough remembers where he was the day Kennedy was shot. Woody Allen sure does. He was in a Los Angeles hotel room writing the script for his first movie, What’s New Pussycat? A chambermaid came in and told him what was going down. “They think he’s dead”, she said. At which point, Allen writes in his memoir Apropos of Nothing, “I turn on the TV and every channel is frantic with the tragedy. I watch for two minutes, digest the information, turn off the set, and go right back to work on my screenplay”.

Allen calls this a “telling anecdote that demonstrates either my discipline and ambition or my lack of connection with reality”. In fact, like Allen’s whole career, it demonstrates both industry and abstraction. “I don’t want reality”, he quotes A Streetcar Named Desire’s Blanche Dubois, “I want magic.” For nigh on 70 years Allen has worked non-stop to fend off the real and to foster reverie.

Writing and directing movies is his way of avoiding real life

Everything he has written — every gag, every short story, every one of his 50 plus movies — is premised on the notion that life is unbearable save for those moments when you are watching Fred Astaire dance or the Marx Brothers play percussion on the helmets of German soldiers.

Nothing can make this engine of comedy seize up — not the assassination of a president, much less an accusation of child molestation. Every writer, Graham Greene famously said, needs a “splinter of ice in his heart”. Allen, this memoir shows, has a nuclear cooling tower in his pigeon chest. Not for nothing, you realise, did he sing that frostiest of love songs, Fud Livingston’s “I’m Through With Love”, in his dream-like musical Everyone Says I Love You: “I’ve locked my heart / I’ll keep my feelings there. / I’ve stocked my heart / With icy, frigid air”.

Apropos of nothing was almost frozen out. Back in March Grand Central Publishing (an imprint of the Hachette group) announced that the book would be coming out on 7 April. A few days later, after a staff walkout in protest at Allen’s being given “a platform [that] validat[es] his story”, Hachette announced it was “return[ing] all rights to the author”. Almost immediately, the book was taken on by Arcade, part of Skyhorse Publishing an altogether smaller outfit.

The book came out in America in late March, to no great fanfare (at the moment it’s available in the UK only in digital form), though Arcade editor Jeannette Seaver did make a statement. “In this strange time,” she said, “when truth is too often dismissed as ‘fake news’, we as publishers prefer to give voice to a respected artist, rather than bow to those determined to silence him.” Amen to that. Apropos of Nothing is no great shakes as literature. But the idea that it might not have appeared merely because its author once again denies allegations that have already been dismissed by the cops and the courts is shaming.

In case you’ve forgotten, in 1992 the then 56-year-old Allen broke off his, ahem, unconventional love affair with Mia Farrow — though they “dated” for years, they “never married . . . never even lived together” — and set up home with one of Farrow’s adopted daughters, the then 22-year old Soon-Yi Previn. A few months later, Allen’s and Farrow’s adoptive daughter, the then seven-year old Dylan, accused Allen of having assaulted her in the attic of Farrow’s country residence.

The Connecticut district attorney investigated and concluded there were no grounds for charges. Since then, a succession of shrinks, nannies and assorted carers, not to mention Dylan’s older half-brother, Moses, have chipped in with further evidence that backs up Allen’s side of the argument. Not that he needs much help. Allen tells us his parents never stopped being disappointed that he’d gone into showbiz and would have preferred it if he’d pursued a proper career. On the strength of the argument he mounts here he’d have made a damn fine lawyer.

Not that he emerges any better from the book than he did from what he calls the “to-do”. While I dare say Allen is on to something when he argues that a vengeful Farrow invented the molestation incident, he still can’t see why almost everyone else goes on looking askance at his hitting on Soon-Yi. To be sure, their affair was no fling. Three decades on the couple are still together — have been married, in fact, since 1997. But people don’t like what he’s done. The audience for his movies has dwindled away. It could be said that there are sound aesthetic reasons for that dwindling. But the bottom line is that Allen simply hasn’t been forgiven for carrying on with a woman 35 years his junior.

Allen affects bafflement at this Scarlet Letter treatment. But really, there is no mystery about it. He makes too many movies in which young babes go for old guys. Granted, as early as 1975’s Love and Death Allen was writing characters (in this case an ancient rabbi) who lusted after the likes of “blonde 12-year-olds — preferably two of them”.

And as early as 1979 (when Allen was 44) he cast himself as the hero of Manhattan, in which his character dated a 17-year-old schoolgirl – and still got to hog the moral high-ground! (Allen has been the subject of so many books and profiles that there aren’t many revelations in Apropos of Nothing. Still, my eyes popped when Allen confessed to having dated a girl of just that age during the shoot for 1977’s Annie Hall.)

But ever since Allen left Farrow for Soon-Yi, his movies have been filled with stories of May to December love – stories whose credibility has steadily diminished as the ages of girl and, er, boy have diverged startlingly. Worse, so many of these movies have been Bildungsfilme in which the girl is schooled in the ways of the world by the old-timer.

Like Allen himself, the heroes of these movies despise rock and roll. Accordingly, they encourage their young charges to listen to Cole Porter, Rodgers and Hart, etc. Fair enough, though to watch the ease with which Allen’s lovers negotiate their cultural differences is rather more grotesque than watching the 76-year-old Mick Jagger bouncing around the stage banging on about “Brown Sugar”.

This is not to dismiss all ripe meets wrinkly romances. But it is to remember that these affairs are rare. That’s why few people pass such a couple without doing a double-take. As Jackie Mason twitted Michael Winner and Jenny Seagrove years ago, “I never seen a girl like that with a truck-driver your age.”

It follows, or anyway ought to follow, that if you’re going to tell stories about what Allen himself calls “unorthodox” relationships you need to dramatise them properly. But it’s been years since Allen dramatised anything properly, years since he shot a script that sounds like anything better than a rough first draft. But then, as Allen more than once admits in Apropos, he has no quality control mechanism.

A gag machine as a kid (earning three times his parents’ joint income by the time he was 18 churning out jokes for newspaper columnists), he has gone on to be a movie machine as an adult, pumping out a picture a year whether or not he thinks it has a hope in hell of being any good.

Allen is very honest about his failures. Every one of his movies has been a disappointment to him. But for all the candour he is too easy on himself. “I don’t have the discipline not to make films,” he told an interviewer back in the 1980s. What he meant, you realise, is that writing and directing movies is his way of avoiding real life.

As this life goes, nobody should be condemned for wanting to avoid it. But a narrative artist like Allen oughtn’t be surprised if his alienation from the world further alienates the world from him. He might not want reality, but as he ruefully concedes, “It’s the only place you can get good chicken wings.”

It’s a great gag, and one that Allen has been wringing variations on all his life. But repeat it though he has, he has never followed up on the joke’s Pirandellian implications. That’s because despite his constant harping on Kafka and Tolstoy and existentialism, he’s not interested in implications. He’s a regular guy, he’d have you know, and he’s interested in what regular guys are interested in. He’s a sports nut, a “Playboy centrefold” fan, “a barbarian sporting the tweeds and elbow patches of the Oxford don”. What he isn’t, he’s adamant, is any kind of intellectual. “I have no insights,” he says, “no lofty thoughts, no understanding of most poems that do not begin, ‘Roses are red, violets are blue.’”

For all the portentous moralising of too much of his recent work, he’s got a point. A school skiver and college dropout, Allen not only isn’t an intellectual, he’s so ill-educated he still believes that important subject-matter makes for important art. Even today, approaching his eighty-fifth birthday, he can’t get his head around the idea that laughter can be as morally bracing as Balzac or Bellow. Like the do-gooding director in Preston Sturges’s Sullivan’s Travels, Allen thinks comedy beneath him. Like Marlon Brando — whom he calls “a living poem . . . who changed the history of acting” — Allen despises his comic talent because he doesn’t have to work at it. He’d rather thrust seriousness at people for whom life is serious enough already.

Since Allen thinks as little of the films that have resulted from this condescending romanticism as he thought of what his fans go on calling his early funny ones, it is difficult to see who has gained from his adolescent loftiness. “Every man wastes part of his life,” said Dr Johnson, “attempting to display qualities which he does not possess.” The greatest comic writer of our era has spent more than half his career trying to convince us that mourning becomes him. Whatever you make of his private life, there is tragedy in that.