Barack Obama’s memoir Dreams From My Father, first published in 1995, played an important role in his progress to becoming President of the United States in 2008. His account of a sometimes troubled childhood, adolescence and early manhood was well received critically but sold a few thousand copies and soon went out of print. But after his election to the US Senate and his electrifying keynote address to the Democratic Party Convention, both in 2004, the book was republished and rapidly became an international bestseller, garnering more critical acclaim over the years.

In the New York Times, Michiko Kakutani called it “the most evocative, lyrical and candid autobiography written by a future president”. Three years ago, Forbes magazine estimated it had earned Obama nearly $7 million. But the original typescript, which has made its way online via a circuitous route, differs in many significant particulars from the published book. While Dreams used pseudonyms for the vast majority of its characters, the manuscript features all of those individuals’ real names. There are also a number of personal, family disclosures that do not appear in the book, and, most strikingly of all, in the early typescript Obama is far more honestly self-critical about his personal shortcomings and insecurities than the eventual book reveals. Additionally, a deeply touching scene involving his Kenyan relatives appears in the manuscript but is absent from the finished book.

Obama had originally envisioned a book entitled Journeys in Black and White. The original typescript reveals that he worked his way through three successive alternative titles before settling on Dreams. Where My Father Lies Buried and Mixing Blood: A Tale of Inheritance eventually shifted to Claims of Inheritance, which were echoed in Dreams’s sub-title: A Story of Race and Inheritance.

Once Obama emerged as a presidential candidate 12 years after his memoir’s publication, journalists and then historians fleshed out the actual identities of many schoolmates, college friends and residents of the Chicago neighbourhood where Obama worked as a community organiser.

The original typescript confirms all of those identities, but it also adds at least a quartet of new names to Obama’s biographical story. In Obama’s Chicago narrative, he develops a close relationship with local community member “Ruby Styles” and her 16-year-old son “Kyle.” The manuscript gives her real name as Harlean Jones, and her son’s as Darrell Jr, but none of the contemporaneous rosters of Obama’s community group include Jones’s name nor do any other participants recall such a person. Obama’s narrative has Darrell turning 16 in May, 1987, but the only African-American Harlean Jones identifiable in multiple web-databases, an Illinois native who died in June 2005 at the age of 48, was born in November 1956, meaning she would have given birth to her son at age 14 if Obama’s account is accurate.

Both Obama’s manuscript and Dreams itself thus may be even more fictionalised than commentators have previously concluded

The typescript identifies Roxanne and Vivian Murdo as sisters “Linda and Bernadette Lowry” who took part in Obama’s organising efforts in the Altgeld Gardens public housing project. Neither sister actually played as major a role as Dreams From My Father describes, but Vivian’s name (unlike Roxanne’s) appears twice in contemporaneous documents.

A deeper mystery concerns an elderly “Rev. Philips” whom Dreams portrays as instructing Obama to introduce himself to the controversial pastor Reverend Jeremiah A. Wright. While Wright and others would later surmise that Rev. Lacey Curry had played that role, Obama’s manuscript uses the name “Rev. Jarrett” notwithstanding the fact that no such minister appears in any records or recollections from those years. Likewise, while Obama’s typescript has a “Rev. Digby” pastoring the largest church in Altgeld Gardens at that time, no Reverend Digby is named in any contemporaneous accounts either. Both Obama’s manuscript and Dreams itself thus may be even more fictionalised than commentators have previously concluded.

Yet the most notable new identity revealed by Obama’s typescript is that of Kenyan historian Theodora Ayot. In Dreams, during Barack’s first trip to Kenya in 1988, his sister Auma takes him to meet professor “Rukia Odero”, who had known his deceased father. Obama’s original manuscript offers a fuller account of his conversation with “Teddy” Ayot than does Dreams, but in both she explains to him how being a good historian “requires a temperament for mischief” and how “truth is usually the best corrective”. In 1996 Professor Ayot moved to Chicago to teach African history at North Park University. She never resumed a close acquaintance with Obama, but in 2008 she donated a total of $240 to his first presidential campaign. Now 73, Professor Ayot retired from North Park in 2018 and still lives on the North Side of Chicago.

But these new identifications are less memorable than several family disclosures that Obama revealed in his initial typescript but then deleted prior to publication. Explaining his maternal grandparents’ view of their pregnant daughter’s situation in early 1961, Obama wrote that “when my mother announced her engagement to my father, both Gramps and Toot tried to talk her out of it.”

Recounting his mother’s upbeat comments about African-Americans throughout his childhood, Barack added in handwriting that “if I got excited about Muhammad Ali, she would take the time to inform me of his stand against an immoral war,” but that addition does not appear in Dreams itself. During his high school years, Obama lived with his grandparents, and while Dreams offers a demeaning portrait of his grandfather, the manuscript reveals a more overtly conflictual relationship, citing “the first time Gramps tried to whip me for some minor infraction and I had struck back”.

Yet it is Obama’s self-revelatory and self-critical comments in the typescript, comments that were removed prior to publication, that are the most weighty differences between the two texts. Speaking of his college years, Obama added by hand that “I spent most of the next four years angry. A curious anger,” but then did not include that statement in Dreams. Come late 1987, when Obama has decided to leave community organising to attend law school, he wrote in his original manuscript:

I’d been feeling run down for some time now. More than run down: I felt defeated. I knew that not much had changed in the neighborhood on account of my presence; that those changes that I had helped bring about were always too partial, too localized, too little too late … No matter what we did, it seemed, our side was always losing ground. It made me edgy, that feeling — the feeling that I had failed my leaders, and failed myself. Still, I tried not to let that feeling show, or even admit it to myself … I didn’t intend to practice law for very long … traditional civil rights litigation didn’t offer much prospect for curing the ills that plagued the South Side.

None of that would appear in Dreams as published, and a few paragraphs later, Obama in his manuscript would go so far as to confess that “Perhaps, once, I had had faith in myself.” In the published book, that statement was revised to read that “Perhaps, still, I had faith in myself.” It would do nothing for an aspiring politician’s reflective self-portrayal to admit in writing that he had “failed” and had lost faith in himself.

Similarly, six months later, as Obama takes a European sojourn on his way to Kenya for the first time, he writes in Dreams, similar to his typescript, that his discomfort in touring Europe had become so intense that “I had been forced to look inside myself and had found only a great emptiness there.” Not in the published book, however, were the two immediately subsequent sentences that appear in the typescript: “The thought had terrified me. Unable to reschedule my flight to Nairobi, I had spent the next two weeks strangely withdrawn, hesitant to talk to strangers, gripped with the irrational fear that everything depended on reaching Africa intact; that with each passing day, my life was slipping away from me.”

However one might choose to characterise or interpret this seemingly profound sense of intrapersonal crisis, it would in no way do a future political candidate any good to admit that he had once experienced such a deep psychological exigency. Once Obama did reach Nairobi, where his sister Auma and his aunt Zeituni Onyango welcomed him, his deep sense of fright evaporated, apparently once and for all. The final chapters of Dreams, recounting Obama’s initial immersion in the Kenyan side of his family he had not previously known, is the book’s most touching and dependable section.

However, at its very conclusion the published book omits a memorable recounting of how at Nairobi airport, after Barack has already cleared security for his departing flight, his father’s stepmother, Granny Sarah, rushes through to hand him his grandfather’s colonial passbook and the carbon copies of his father’s old application letters to US universities. Her remarkable gesture exemplified his Kenyan relatives’ deep desire to draw their impressive American offspring into the family bosom, but Dreams does not tell readers about this poignant moment.

Dreams as published nonetheless ends with a two-page paean to Barack’s older brother, Abongo Malik, and his impressive presence at Barack’s October 1992 wedding. Rather than his new spouse, Michelle, “the person who made me proudest of all” was Malik, who “looked so dignified … that some of our guests mistook him for my father”, Barack wrote. “He was certainly the older brother that day” and Barack celebrated how “we can disagree without rancor” because Abongo Malik’s “heart is too generous and full of good humor, his attitude toward people too gentle and forgiving”.For Barack Obama to reread those words now must be painful indeed, for the tale of how the initial typescript first came into public view is a terribly sad story of how the once close relationship between the two brothers became totally sundered in 2014-15 during the second term of Barack’s presidency.

On 7 April, 2014, Zeituni Onyango died aged just 61 in Boston, Massachusetts, where she had been living a penurious existence in high-rise public housing. Her insistently upbeat demeanour had belied her modest circumstances, and she never spoke ill of her world-famous nephew despite his utter lack of interest in the recurring challenges she had faced. The New York Times reported that the president “helped pay funeral expenses” but “did not attend, as he was golfing”.

Yet Zeituni’s death also forced the Obama clan to tackle how to pay for returning her body to Kenya for burial, and, in Malik’s telling one year later, when he visited Barack at the White House to ask for his assistance, their conversation did not go well. “I told him that … she loved you very much and we need … around twenty thousand dollars and he said that was too much.”

How accurate, or inflated, Malik’s estimate may have been is unclear, but Barack’s response deeply angered his older brother. “I told him, ‘You say you’re your brother’s keeper. I don’t feel it, and I don’t see you living up to what you say.’ She had been really good to him” back in 1988, and “I felt really sad that he would just abandon her like that. I just left. She was stuck there for a month” before transport from Boston to Nairobi was secured.

Malik’s fury at his younger brother did not dissipate, and indeed within just a year it had festered to the point that Malik willingly accepted the eager embrace of a handful of “birthers,” conspiracy theorists obsessed with challenging Barack Obama’s life story. In April 2015 Malik gave a Skype video interview to director Joel Gilbert, whose 2012 film Dreams From My Real Father purported to document the outlandish notion that Barack’s actual biological father was the late African-American poet and one-time communist Frank Marshall Davis, a good friend of Barack’s maternal grandfather in Honolulu and a self-professed adventurer in all things sexual.

Malik’s comments about his younger brother were as heartfelt as they were devastating. “The way that he’s turned and become a different person with the family” was at the heart of their breach, as their argument over Zeituni’s burial had highlighted. Malik was upset with “how he treats people”, and he asserted that among Barack’s Kenyan relatives, “deep down inside everybody is really disappointed and upset and angry”.

He revealed that he had an early typescript copy of the manuscript that had become Dreams From My Father, explaining that Barack had given it to him so that “I should go through the book just to make sure that everything was OK” with what it said about their Kenyan family history. Malik doubted some of what Barack had written about their grandfather’s experiences with British colonisers, but he confessed to Gilbert that Barack “doesn’t want anything to do with me anymore”.

But irrespective of gilbert’s track record, Malik’s comments to him displayed sadness as much as anger. How can Barack “be so cold and ruthless, and just turn his back on the people he said were his family?” Malik asked out loud. “It’s hard to understand how somebody can make such an about-turn and make a big play about where he comes from and then once he gets what he gets, wants nothing to do with that place anymore.”

Gilbert was eager to entice Malik into endorsing his Frank-Davis-as-Barack’s-father craziness, but Malik’s descent into birther-world did not end with his long-distance interview with Gilbert. Two years earlier Malik had already sold to an autograph dealer two inconsequential White House notecards signed by Barack, but now he offered up his full trove of invaluable family heirlooms, including mildly newsworthy handwritten missives from Barack dating from years earlier.

By June 2016 Malik’s copy of Barack’s hand-annotated Dreams typescript had been purchased for $7,500 by conservative journalist Charles C. “Chuck” Johnson, who hoped that it might somehow advance the far-loony claim that Dreams had been secretly ghostwritten by former student radical Bill Ayers, a Chicago neighbour whom Barack was just beginning to befriend when Dreams was first published.

In months to come a video copy of Auma Obama’s 19-minute 1993 home movie, A Journey in Black and White, filmed during Barack and Michelle’s 1991 visit to Kenya, made its way onto YouTube thanks to a far-right British conspiracy theorist, Paul Joseph Watson. Eventually Malik would even sell one of his father’s Kenyan passports to Chuck Johnson. Covering the period from September 1964 until February 1972, all of the visas, entry and exit stamps in Obama Sr’s passport match up 100 per cent with the known facts of his life and travels, thus offering no fodder to the birther community.

When Johnson first announced his acquisition of Malik’s copy of the early Dreams typescript in late June 2016, no one in mainstream journalism took any notice, but by 13 July a 566-page PDF of Johnson’s acquisition was visible via Johnson’s now-defunct GotNews website and by 20 July via a now-long-inactive Norwegian language WordPress blog. Johnson’s PDF, adorned with several dozen yellow sticky notes of indeterminate origin, unfortunately was 14 pages short of a complete copy, with five gaps of one or more pages readily apparent to anyone who made the effort to download and examine it closely.

But apparently no one did, and Johnson’s costly acquisition quietly gathered web-dust until in July 2019 Johnson took the initiative to bring it to the attention of this duly-embarrassed Obama biographer, who had given up paying even a modicum of attention to Ayers- and Davis-obsessed conspiracy theorists well before summer 2016.

But the larger irony is how the birthers’ fixations on their Ayers and Davis fantasies precluded them from examining Obama’s typescript with sufficient care to appreciate that there was real news in Johnson’s purchase — just not the sorts of news he was hoping to find. Obama’s deleted admission that he felt he had “failed” in his community-organising work, a far different verdict from the successes he would claim as an aspiring presidential candidate, gives important new insight into the resumé-polishing purpose of Dreams.

Likewise, Obama’s redaction of the depth of the psychological crisis he had experienced in Europe in summer 1988 — that he experienced an “irrational fear” that “my life was slipping away from me” — would certainly have drawn probing questions from journalists curious about a difficult-to-fathom moment in an incumbent president’s life.

Obama’s initial typescript may not fundamentally alter commentators’ critical appraisals of Dreams, but it does significantly enrich our historical understanding of a president whose life story will continue to attract widespread attention for years to come.