Donald Trump “promised an elixir for the racial anxiety” of “millions of Americans spooked by a black man in the White House”, Barack Obama writes in his eagerly awaited memoir.

Those Americans, Obama writes, were prey to “the dark spirits that had long been lurking on the edges of the modern Republican party – xenophobia, anti-intellectualism, paranoid conspiracy theories, an antipathy toward black and brown folks”.

In A Promised Land, which comes out on Tuesday, Obama continues: “It was as if my very presence in the White House had triggered a deep-seated panic, a sense that the natural order had been disrupted. Which is exactly what Donald Trump understood when he started peddling assertions that I had not been born in the United States and was thus an illegitimate president.”

Penguin Random House reportedly paid the former president and his wife, Michelle Obama, $65m for books about their time in the White House. The former first lady’s memoir, Becoming, came out in 2018 to widespread acclaim.

Excerpts of Obama’s book have run in the press – the remarks above were reported by CNN – and the former president is due to speak to CBS in two interviews on Sunday. The New York Times has also run a lengthy review. The 768-page volume is the first of two, covering Obama’s rise to the US Senate and then the White House as the 44th president, from 2009 to 2017. It has been a struggle to write.

“I figured I could do all that in maybe 500 pages,” Obama wrote in an excerpt published by the Atlantic on Thursday. “I expected to be done in a year. It’s fair to say that the writing process didn’t go exactly as I’d planned.”

Obama also says he is “painfully aware that a more gifted writer could have found a way to tell the same story with greater brevity (after all, my home office in the White House sat right next to the Lincoln Bedroom, where a signed copy of the 272-word Gettysburg Address rests inside a glass case)”.

A Promised Land heads for the shelves as Trump refuses to concede a clear electoral defeat by Joe Biden, Obama’s vice-president, deepening dangerous political divides.

Obama considers Trump’s rise, from reality TV host and political gadfly, champion of the “birther” lie which held that Obama was not born in the US, to outsider candidate, GOP nominee and norm-shattering president.

Obama recalls his first presidential election and the storm over his healthcare reform, the Affordable Care Act (ACA), two years later. He echoes many observers in detecting the roots of Trumpism in the surprise rise of Sarah Palin, the Alaska governor who became John McCain’s running mate in 2008 and two years later fanned the flames of the Tea Party, the rightwing movement which railed against the ACA.

“Through Palin,” Obama writes, “it seemed as if the dark spirits that had long been lurking on the edges of the modern Republican party – xenophobia, anti intellectualism, paranoid conspiracy theories, an antipathy toward black and brown folks – were finding their way to centre stage.”

Obama wonders whether McCain would have picked Palin had he suspected that “her spectacular rise and her validation as a candidate would provide a template for future politicians, shifting his party’s center and the country’s politics overall in a direction he abhorred.

“I’d like to think that given the chance to do it over again, he might have chosen differently. I believe he really did put his country first. We’re better than this.”

Reviewing Trump’s rise to power, Obama considers how Trump seized on a growing inclination among Republicans to dispense with evidence and polite political convention, in the name of simply opposing the first black president.

“In that sense,” Obama writes, “there wasn’t much difference between Trump and [House speaker John] Boehner or [Senate majority leader Mitch] McConnell. They, too, understood that it didn’t matter whether what they said was true … in fact, the only difference between Trump’s style of politics and theirs was Trump’s lack of inhibition.”

As the Biden presidency approaches, Republicans seem likely to hold the Senate. Among Democrats, much hope of legislative progress rests with how the new president will be able to deal with the notoriously hardline Senate leader.

Obama writes that he chose Biden as his emissary to McConnell in part because of his own “awareness that in McConnell’s mind, negotiations with the vice-president didn’t inflame the Republican base in quite the same way that any appearance of co-operation with (black, Muslim socialist) Obama was bound to do”.

Obama discusses his famous roast of Trump at the White House Correspondents’ Association dinner in 2011, on the same night a Navy Seal team was preparing to find and kill Osama bin Laden. He also details two surprising offers of help from Trump – to plug the Deepwater Horizon oil well, in 2010, and to build a pavilion on the White House lawn. Both were turned down.

In the Atlantic excerpt, an adaptation of the preface to A Promised Land, the former president comments on the 2020 election, during which he campaigned for Biden.

“I’m encouraged by the record-setting number of Americans who turned out to vote,” he writes, “and have an abiding trust in Joe Biden and Kamala Harris, in their character and capacity to do what is right.

“But I also know that no single election will settle the matter. Our divisions run deep; our challenges are daunting.”

But at the end of a year marked by national protests for racial justice, Obama’s thoughts and comments about race and his presidency will no doubt earn particular attention. At one point, CNN reported, he writes of watching television with his wife Michelle, and catching “a glimpse of a Tea Party rally”.

“She seized the remote and turned off the set,” Obama writes, “her expression hovering somewhere between rage and resignation. ‘It’s a trip, isn’t it?’ she said … ‘That they’re scared of you. Scared of us.’”