Four years ago, Paul Simon had an unusual dream. “You are working on a piece of music called ‘Seven Psalms’,” said the voice. For months, Simon woke frequently in the night, scribbling down words recovered from a succession of similar dreams. Turned into lyrics and set to music, Simon released an album — the fruition of that work — in May. In a new documentary about his career, In Restless Dreams, he describes the making of the album as a “journey for me to complete”. Reflecting on its concerns about death, spirituality and the nature of God, he said, “This whole thing is really an argument I’m having with myself about belief — or not.” He was talking about Seven Psalms, but “this whole thing” might well apply to his sixty-year-long journey in music.

Even in the Simon & Garfunkel years, the signs were there

Paul Simon grew up in Queens, New York, the first-born child of Hungarian-Jewish parents. Paul and his brother had their bar mitzvahs and their mother attended synagogue on the high holidays, but the family was otherwise non-practising. It was instead baseball and what Simon would call that “deep forbidden music” that moved his soul: doo-wop groups like The Penguins, The Moonglows and The Five Satins; Elvis Presley’s sensual fusion of country and blues; and the haunting sounds of gospel, beamed into his bedroom from southern radio stations late at night.

In the beginning, in his musical partnership with his childhood friend Art Garfunkel, there were few of the spiritual themes that would define his later work. “I am a Rock” is about angsty social alienation. “The Sound of Silence” (“Hello darkness, my old friend”), the song that made him famous, captured the mood of a country still reeling from the Kennedy assassination. “The Boxer”, about a young man battered and bruised by life’s setbacks and frustrations, reflects Simon’s anxiety about that growing fame.

Even in the Simon & Garfunkel years, though, the signs were there. The plain language of “The Boxer”, Simon later claimed, was lifted from a Bible chanced upon in a hotel. On “Mrs Robinson”, theme song for the film The Graduate, Simon sings like an exuberant street preacher (“Jesus loves you more than you will know”). On “Cecilia”, he appeals to the patron saint of musicians, comparing the divine intervention of his muse to an afternoon of love-making. It was only after his split with Garfunkel in 1970 that spiritual themes came to the fore in his work, though. His management feared the decision would restrict him commercially. In fact, it freed him to pursue his own artistic search.

“Duncan”, a song from his self-titled 1972 album, might serve as a metaphor for that search. Young Lincoln Duncan, a fisherman’s son, leaves his home behind “without a penny in [his] pocket”, bound for “sweet New England”. In a parking lot, he comes across a young woman — a preacher — “singing sacred songs and reading from the Bible”. She seduces Lincoln, who afterwards lies enraptured in a “garden of delight”, “playing [his] guitar / Lying underneath the stars”. His fulfilment is short-lived, however. He narrates the story of the sexual encounter from a cheap motel, and his plaintive “I know, I know”, which follows the telling, suggests whatever he is searching for has yet to be found.

Throughout the 1970s, as a solo musician, Simon continued his Duncan-like artistic search. Some of the spiritual themes that would define his later work emerged in this period. On “Silent Eyes”, he mystically imagines Jerusalem “burn[ing] like a flame” and sings that “We shall all be called as witnesses/ To stand before the eyes of God”. On “Slip Sliding Away”, he muses on the transience of life. By the beginning of the 1980s, he was bound for the motel of midlife disappointment. His marriage to actress Carrie Fisher had ended in divorce; his sixth solo album Hearts and Bones was a flop.

In the midst of a depression and “looking for a shot of redemption”, he began listening to tapes of mbaqanga, the pop music of South African townships. It rekindled his creative spark. His artistic search became a literal one, when he travelled to Johannesburg to record with local black musicians the tracks that would comprise Graceland. Though the music is South African, Simon’s lyrics reflect his own preoccupations. The album’s title track, for instance, tells the story of a divorced Dad, driving his child through Mississippi to Graceland, the home of Elvis Presley. They hope, like the other “poor boys and pilgrims”, to be received by the spirit of the king of rock n’ roll: a metaphor for Simon’s hopeful return to the Edenic garden of his creativity.

To live in the questions is to be alive to possibility

Rhythm of the Saints, his follow-up album, struck a similar note. This time, however, Simon’s artistic search took him to Brazil, where he recorded a ritualistic drum music used in an African diasporic religion native to the country. Once more his lyrics reflect his own preoccupations. On “The Obvious Child”, an ageing everyman reflects life’s burdens are a manageable weight (“the cross is in the ballpark”). On “The Coast”, a troupe of musicians take shelter in the church of Saint Cecilia — Simon’s patron saint — to praise a “soul’s returning to the earth”. Like Christ, artistic resurrection is possible, Simon seems to say. His lyrics are inspired by the work of his friend, the Saint Lucian poet Derek Walcott, whose writing “never separated poetry from prayer”.

In recent years, Simon’s spiritual themes have become even more overt. So Beautiful or So What, released in 2011, includes “Afterlife”, where a man comes face to face with God. “So What”, meanwhile, meditates on the meaning of life. These same questions return on Seven Psalms in which Simon seeks new metaphors for God: “the Lord is my engineer”, “the Lord is the earth I ride on”, “the Lord is a virgin forest”. On “Your Forgiveness”, he expresses his hope that heaven’s “gates won’t be closed” to him, before giving his reasons to doubt: “two billion heartbeats and out/ Or does it all begin again?”

As Simon has aged, the questions have multiplied, his spiritual search never calcifying into belief. That distinguishes him from some of his contemporaries. Bob Dylan, also born to Jewish parents, became a born-again Christian in the 1970s. He released a trio of gospel albums and warned of the second coming. Leonard Cohen approached religion with a voracious appetite. In his own words, he “tried everything”, including five years as a Buddhist monk. Whilst Cohen’s life and career were (like Simon’s) a spiritual search, he never abandoned the Judaism of his birth. “I’m not looking for a new religion,” he once told an interviewer. “I’m happy with the old one.”

Simon has never reconnected with his Jewish roots in this way. He has, instead, talked of spirituality as “something [he] enjoys”. Perhaps that is instructive. The Austrian poet Rainer Maria Rilke, whose work emphasises an intense communion with life, stresses the importance of “living in the questions”, rather than demanding answers we would not understand were we to receive them. Simon has never stopped asking questions, which has pushed himself and his music to new places. In Restless Dreams, he talks of his artistic search as reaching for something that is beyond the horizon of what he can hear, something that might not even exist.

To live in the questions is to be alive to possibility. In Sickness Unto Death, the philosopher Søren Kierkegaard muses, “God is that all things are possible.” Not only does God make all things possible, but possibility is God. If all things were certain, or if life were confined to the here and now, it would be dull and boring. Possibility, and our desire to actualise it, makes life worth living. For Simon, music — and his search for it — surely represents this possibility. On Seven Psalms, he is explicit. The Lord, he sings, is his “record producer”. That may read as tongue-in-cheek. Sung by Simon, though, it rings true. As followers of his journey in music, we too may catch a glimpse of that unreachable thing beyond the horizon. You might even call it “Graceland”.