The decimation of Indigenous people and the enslavement of Africans are linked. America’s entire saga of white supremacy is a legacy of our English heritage. It’s no surprise that we have a history of systemic police brutality, harassment and bias as disheartening as any bedeviling London or far-off Brisbane.

With his new book, subtitled A Story of Violence, Injustice and the American City, Nicholas Dawidoff explores one chilling example from his home town, New Haven, Connecticut. It was your typical frame-up of an innocent inner-city youth. Naive, unworldly and eager to return home from hours of interrogation, Bobby Johnson, 16, rashly believed a detective, Clarence Willoughby. Questioned with no one else present, Johnson was strong-armed. Told that if convicted “he was liable to be executed”, the young man was promised that confession would mean probation. And so, three times, in three different ways, an innocent adolescent admitted murder.

New Haven is best know as the home of Yale, another colonial relic, a place suffused in privilege amid a sea of want. The casual calamity occurred on a still summer’s evening in 2006. What happened was not unusual, just another confrontation among the raging rows of race, class, guns and inequality that beset urban ghettos. Ultimately, such bleak places offer neither opportunities to break free nor ready, legal means of sustenance. Leaving residents hopeless, often at each other’s throats, ghettos allow for wholesale victim-blaming of defeated people deemed inhuman.

In such a quarter, during a hold-up, Herbert Fields, a middle-class 70-year-old Black grandfather, was shot in his car at point-blank range. All his life, Fields had worked hard and done well. Due to tenacity and the good luck to retire before the last manufacturers left town, he had even managed to escape the Newhallville slum.

On frequent returns, he hung out, helping friends and former co-workers who hadn’t made the break. Many were from families like his. They had made the Great Migration from South Carolina, seeking well-paid factory jobs. Fields gave, loaned and flashed cash on the first of each month. That day in 2006, he was carrying more than $1,500.

The alleged killer, Johnson, was trapped with the rest of his family in Newhallville. Following a rapid-fire investigation, tainted by evidence faked and withheld by dishonest officers, he did not get probation. He got a 38-year sentence.

Dawidoff, an Art for Justice fellow and Pulitzer prize finalist, returned home from New York to spend eight years sorting out the harsh truths of the case. He interviewed more than 500 people. For all his painstaking journalistic rigor, the resulting book is hardly academic or tedious. Dawidoff puts the reader at the scene with vivid prose and attention to detail. Not unlike Truman Capote’s In Cold Blood, this is a book as gripping and fast-paced as a bestselling fictional mystery.

Some are certain to welcome the author’s strategic editing of names and lack of illustrations, moves which contribute to the mythic vibe. Some may even feel it helps to elevate this work into a more identifiable, universal and cautionary tale. However, to me, such omissions, though well-intentioned, are the chief flaw of The Other Side of Prospect.

Honoring Johnson’s request, Dawidoff said he agreed to only identify him in the text as “Bobby”. Similarly he omits the amount, just shy of $3m, of the settlement the city of New Haven eventually paid. After nine years in prison, followed by exoneration and a successful lawsuit, Johnson sees his restitution not as a windfall but as “reparations”.

Besides a dedicated local advocate, the New Haven attorney Ken Rosenthal, appointed to represent him in 2010, and New York attorneys associated with the Innocence Protect, what ultimately served Johnson best was the incompetence and detachment of everyone who ought to have protected him. The detectives who charge him ignored quite promising evidence that very likely would have led to the real killer. They failed altogether to examine video from the Visel Pharmacy. It proved Johnson’s assertion that when Fields was killed, he was there.

Eventually, the inquisitor-like Willoughby was arrested for corruption and for allegedly having framed many others. In 2009, he was found not guilty.

Dawidoff portrays the snuffed life of Herbert Fields with the same care he devotes to the unpromising progress of Fields’s likely killer, Larry Mabery. But the author is at his best examining Johnson. His depiction of alienation greater even than the ghetto’s is probably as true a slice of prison life as you’ll ever read.

“If you don’t have family [to put money in your commissary account], you don’t have soap.”

Prison’s “horrible stink”, Johnson stresses, eventually “became natural”. Laundry was unreliable. White clothes came back stained. Johnson took bird baths in his cell’s sink. He began to clean his clothes there too. When the power failed due to jerry-rigged electric cookers, there was “literal darkness … for 13 hours, until the daylight seeped in through the window crevices”.

Putting into context Johnson’s relocation and mostly happy resolution, Dawidoff writes of a reflective wiser person, who says: “I was listening to the tape the other day … thinking how deep and intimate it was.”

To this, his chronicler responds: “It didn’t cross his mind, he’d just been fleeced of his own story. A coerced, false confession could be considered a metaphor for how Bobby grew up. He had no agency. He’d become who somebody else expected him to be.”

“Painting” Bobby Johnson’s portrait with just words, with missing facts and without photographs or an index, also detracts from agency. It’s all a diminution of his individuality, even his humanity. But it is not enough to recommend avoiding this fine work. Such are its merits, it seems sure to become the basis of a film, which will lend this story even more immediacy.

• The Other Side of Prospect: a Story of Violence, Injustice, and the American City is published in the US by WW Norton & Company