In fewer than 400 pages, Clint Smith’s debut work of non-fiction is an intrepid trek covering lots of ground.

Engaging our nation’s “curious institution” – “our un-atoned original sin” – the book purports to examine just eight sites.

New Orleans, the poet-author’s home town; Jefferson’s Monticello; the revisionist tourist attraction at the Whitney Plantation; the infamous maximum security prison at the 16,000-acre Angola Plantation; Blandford Cemetery, a Confederate shrine with Tiffany windows; Galveston Island, commemorating Juneteenth; New York City, enriched by slave-based enterprise; and Gorée Island, with its gate of no return, in Senegal.

All are stops on Smith’s pilgrimage. But because of his scholarly curiosity and zeal, his quest includes many more.

This is an elegiac discourse, sometimes a trifle overwrought. No matter. It answers the title’s question with often elegant emphasis. In a time when youth of color, despairing of the elusive American Dream, seek Wakanda instead – with many whites afraid to acknowledge either privilege or disparity and the role, since the beginning, slavery has played in both – the way “the word” about our history has been passed is with half-truths and delusion, but most of all through denial.

Smith “had never been taught that the largest slave rebellion in US history happened just miles from the city that had raised me … that the Louisiana Purchase was a direct result of the Haitian Revolution”.

Colossal, sculpted to endow Robert E Lee with an appearance of valor no captioning could rectify, the ignoble commander’s New Orleans monument is gone now. But noting that “at least a hundred streets, statues, parks, and schools named after Confederate figures, slaveholders, and defenders of slavery remain”, Smith suggests the task of eliminating symbols which hurt and humiliate is only just beginning.

Thomas Jefferson’s aerie at Monticello, Virginia, attests to how those so invested in maintaining Confederate memorials might consider that even antebellum architecture or old silver can be imbued with the taint of slavery’s oppression.

Persisting in keeping his wife pregnant though aware how pregnancy, which ultimately killed her, had put her life at risk, what sort of man was our third president? He demeaned African Americans as inferior and defamed black women as unattractive even to black men. Yet Jefferson maintained a decades-long relationship with an enslaved girl, begun when she was 13 and he was 43.

Enslaving 600 people, this was the man who proclaimed that all men are created equal, that: “Indeed I tremble for my country when I reflect that God is just, that his justice cannot sleep forever … these people are to be free.”

Though he denied the intelligence and ability of all black people, Jefferson would have had very little without them. Relying on testaments from former slaves, Smith tells of one man who stated, to Roosevelt’s Works Progress Administration in the 1930s, how he knew better than the libel of men like Jefferson.

We thought we was goin’ to be richer than the white folks, ’cause we was stronger and knowed how to work, and the whites didn’t and they didn’t have us to work for them anymore … We soon found out that freedom could make [black] folks proud but it didn’t make ’em rich.”

All save one of the rudimentary slave cabins and workshops along Mulberry Row that supported the luxury of Monticello have vanished. The Whitney Plantation, in Louisiana, presents a different story. To Smith, “It is a hammer attempting to unbend four centuries of crooked nails. It is a place asking the question, ‘How do you tell a story that has been told the wrong way for so long?’”

Whitney is as much a series of memorials to slaves as an exhibition about slave life. Some find it lacking in historical integrity. This is due to augmentations. One is an 1890s Queen Anne-style shingled black Baptist church, with lifelike statues of abandoned black children. There are prison cells from the early 1900s, salvaged from Pennsylvania. Smith qualifies these embellishments as perhaps “a necessary, even if imperfect, corrective against a history that has been misrepresented or ignored for so long, a place that does far more good than harm”.

Referring to the penitentiary at Angola and its history of horrors, Smith laments, “If in Germany today there were a prison built on top of a former concentration camp, and that prison disproportionately incarcerated Jewish people, it would rightly provoke outrage throughout the world.

“… White supremacy … numbs a whole country – black and white – to what would in any other context provoke our moral indignation.”

Smith’s closing segment in Africa, where he discovers schoolgirls more knowledgable about the trade than most Americans, offers real hope.

Conversely, in New York his explorations inspire disappointment. He begins at the African Burial Ground, which segregated blacks outside the city walls starting in the 18th century. “These bones belong to us,” opined the chair of the landmarks commission in 1991, after the remains were unearthed.

But Smith’s visit to the site of Seneca Village, once a rustic Manhattan hamlet of 225 residents, 150 of African descent, is of more ambiguous merit. In 1860, 10 of those blacks owned their houses and so could vote: 10 of approximately 12,000 black New Yorkers, out of 813,669 inhabitants in total. Was it worth it to lose Seneca Village and gain Central Park?

Even knowing that some of those 10 black men were compensated at a rate below the true value of their property, I think so. More troubling by far is gentrification and displacement today. It has made the Treme, Washington and Harlem, with all the attainment and history each embodies, today’s Seneca Village. These were communities of color far richer than Greenwood’s Black Wall Street. Bloodless, their evisceration is no less thorough than the result of Tulsa’s race massacre.

As one knows better through this welcome work, that is how the word is passed.

• How the Word is Passed is published in the US and in the UK by Little, Brown