In the midst of the civil war, when the battle was not going well, Abraham Lincoln wrote that “[w]e must disenthrall ourselves, and then we shall save the country”.

Harvard professor Jill Lepore chooses to begin her history of the United States with that quotation, and much of the worst of America, from lynching to brutality to Native Americans, is rightly here. But her true purpose is much broader: as she writes, the constitution adopted in 1787 was meant to determine whether government could rule “not by accident and force but by reason and choice”.

In America’s other founding document, the declaration of independence, Thomas Jefferson wrote: “We hold these truths to be self-evident, that all men are created equal, that they are endowed by their Creator with certain inalienable rights, that among these are life, liberty and the pursuit of happiness.” Can this be true, for everyone of whatever race and for women too? Is it possible for the US – or any nation – to be ruled by reason and choice? For Lepore, these are the essential questions.

The “constitution cannot be made easy”, she writes, because “it was never meant to be easy”. Moreover, she notes that though the United States was “founded on a set of ideas … Americans have become so divided that they no longer agree, if they ever did, about what those ideas are, or were”. This is, therefore, a history of political equality which necessarily becomes primarily a political history.

It is also, very largely and appropriately, a history of race in America, of the attempts of many to realize “these truths” and the heartbreaking struggles of those who have been oppressed. This began at the beginning. From 1619, “liberty and slavery became the American Abel and Cain”. The question nearly sundered the colonies from all government. When George Washington died in 1799, Lepore writes, “the black people in that room outnumbered the white people”. But his will, published throughout the country, delayed emancipation of his slaves until after the death of his wife.

In a nation devoted to freedom, why were some free and others not? Like so many Americans, Lepore asks that question and another: “By what right are we ruled?”

Her aims are ambitious. This is also a history of journalism, including what may be the first real history of the internet in a political context. It is also a history of technology and a history of religion, which has influenced so much American life and thought. Finally, “this book aims to be something else, too – an explanation of the nature of the past.”

“History isn’t only a subject,” Lepore writes. “It’s also a method. My method is, generally, to let the dead speak for themselves.”

Lepore therefore gives prominence to figures such as Caesar, a slave hanged for rebellion in colonial New York; David Walker, a ship chandler who founded the first African American political organization; Mary Lease, a leading figure in the populist movement of the 19th century; and Ivy Lee, a pioneer in public relations, for good or ill. She tells new stories, such as that of Harry Washington, an escaped slave from Mount Vernon who went to Canada and then to the new colony of Sierra Leone, or of southern women rioting for bread during the civil war who thereby helped lay the foundation of a public welfare system.

It is the story of a nation, multiracial at its founding, and those who sought to find ways to realize “these truths”. In the 1800s, Frederick Douglass admired the new art of photography because it represented faces perfectly rather than through the subtle distortions of painting, and could therefore help African Americans speak for themselves. A century later Dr Martin Luther King Jr realized that the constitution could be on his side, because “the great glory of American democracy is the right to protest for right”.

American politics has always been robust, but technology and better methods of analysis have magnified the impact. Lepore entitles her section on recent history “The Machine”: a double entendre for the political “machines” of the past.

She introduces Clem Whitaker and Leone Baxter, the founders of Campaigns Inc, one of the first modern consulting firms. Its successes were vast, from the defeat of health insurance proposals in California to the election of Richard Nixon, and its influence on future campaigns even more profound. As Lepore writes: “The Like Factor … came to drive American political communication, decades before the rise of Facebook … in effect, the Like Factor replaced the Fairness Doctrine.” Polling undercut the views of minorities and overweighted majority views.

Democrats also played a role, as in the Kennedy campaign’s use of new “data science” in 1960, an election which Lepore notes “may have been rigged”. Lepore also shows how liberal public opinion shifted in the 1950s to become “contemptuous” of mass culture, just as conservatives had been in the 30s. Franklin D Roosevelt was the last leader truly to unite rural populism and urban activism.

Given all this, Lepore acknowledges that the US “had endured eras of heightened partisanship before … beginning in the 1990s [it] started a long fall into an epistemological abyss.” Even Leone Baxter, towards the end of her life, decided she was “worried about men like [Roger] Ailes”.

It’s not quite accurate to say that this book is Lepore’s response to or explanation of the election of Donald Trump, but that event constantly lurks in the background, from a discussion of Joseph Pulitzer’s quest for accurate journalism based on facts to meditations on the forces of populism, reaction and nativism throughout American history.

Lepore also links Trump firmly to the deleterious – and often anti-democratic – technological and campaigning innovations of the last 50 years: “Polls admitted Trump into the GOP debates, polls placed him at center stage, and polls declared him the winner … Time posted this warning: ‘The results of this poll are not scientific.’ Less reputable websites did not bother with disclaimers.”

There is always a debate about where politics ends and history begins. Lepore draws the line generously in favour of history. The book is a first draft of the 21st century – nearly 100 pages consider America after 1992. It is hard to reach judgments about that period, though Lepore links current events to their origins in the distant past: the populism of Andrew Jackson, how the populist William Jennings Bryan strongly opposed social Darwinism.

She offers an unabashedly liberal perspective, but seeks to be scrupulously fair to the modern conservative movement, devoting numerous pages to its intellectual origins as well as to its nativist and conspiratorial elements. Ideas do have consequences, as wrote Richard Weaver, a conservative intellectual for whom Lepore has sympathy.

This is a history for the 21st century, far more inclusive than the standard histories of the past. Is Lepore hopeful? Recall the quote with which she opened. Lincoln did not say merely that we “can” save the country, but that we “shall”. The Lincolnian spirit – and a fair dose of civics – will do much to restore the values of the American republic.