In 2015, Monica Potts reconnected with an old friend named Darci on Facebook. The two of them had grown up in Clinton, a small rural town on the edge of the Ozarks in Arkansas, and had been best friends in high school. In their teens, both vowed they would leave and build a life away from the dysfunction and deprivation of their home town, but for Darci it didn’t pan out like that. While Potts, who is now in her 40s, gained a scholarship to a college in Philadelphia, and became an award-winning political journalist (she has written for the New York Times and the Atlantic, and is now senior politics reporter for FiveThirtyEight), Darci failed to graduate high school and, like many in their social circle, never left Clinton. Over the years, Potts heard on the grapevine that Darci was struggling. When she finally visited her friend on Christmas Eve, 2015, she found her in crisis: separated from her two children, battling addiction, living in a trailer with a man she barely knew and stuck in a repetitive loop of jail time, release, decline and rearrest.

At the time, Potts was already investigating a disturbing trend: in 2012, a team of population health experts at the University of Illinois Chicago had found that white women who failed to graduate high school were dying five years earlier than the generation before them. Subsequent studies revealed a sustained fall in life expectancy among the least educated white American women, and an increase in so-called “deaths of despair”: drug overdoses, suicide and complications from alcoholism. Further research in 2018 by economists at Dartmouth College also revealed rising numbers of deaths from cancer, heart disease and respiratory illness among women below the poverty line. One of the researchers, Charlie Rafkin, told Potts that women suffered from a range of disadvantages, making it hard to pinpoint a single cause for the mortality increase. Instead he attributed it to a “general malaise”.

Potts wanted to know more about this malaise and to hear the stories that can’t be told with data and statistics. “This, I felt, was where the work of sociologists and economists and public health experts ended and my work as a journalist began.” And so The Forgotten Girls was born, an insightful, detailed and moving blend of memoir and reportage revolving around Potts’s home town of Clinton, “with its ageing, shrinking population, governed by a small group of people who worshipped at the same churches as their parents and who had knit around themselves an even thicker and tighter web of personal and political self-deceits”.

Her aim is to understand why she was able to fulfil her dream while Darci got sucked into a vortex of poverty, addiction and relationships with feckless men. If that sounds like an exercise in condescension – see how I succeeded where my friend failed! – Potts is never less than respectful, presenting Darci’s story as commonplace among her rural peers and her own flight from Clinton as a deviation from the norm. Potts talks to family members, neighbours and old school peers to build a picture of the lives of rural Arkansans, their religious beliefs, politics, relationships, employment prospects and the influence of external factors such as the 2007-8 financial crisis and Covid-19. But the book’s backbone is Darci, a charismatic, mercurial and ultimately exasperating figure who, along with her fellow Clinton natives, has been ‘‘buffeted by the same forces that troubled the entire country”.

Potts is by no means the first writer to shine a light on the plight of the white American underclass. The rise of Donald Trump and his election by mostly non-college educated white voters highlighted the insecurity and insularity of this demographic, and led to a wave of books including 2016’s White Trash by Nancy Isenberg, which traced 400 years of class division in America, and law professor Joan C Williams’s White Working Class in 2017. By providing an insider-outsider view, Potts’s book ostensibly has more in common with Hillbilly Elegy, JD Vance’s 2016 bestselling memoir, recently turned into a film by Ron Howard, which detailed the author’s journey from rust-belt Ohio to Yale law school and the Ohio State Senate. But where Vance, a Republican, railed against welfare recipients and a culture that encourages victimhood and failure, Potts takes a broader, more compassionate view, underlining the invisible forces that can shape people’s futures and those of their children.

None of this is to suggest the people of Clinton are without agency. There is a lot of talk among Potts’s interviewees of individuals making poor choices. Darci’s decision to party during the final years of high school led to her not graduating, which, Potts notes, “instantly put her in one demographic group, and me in another”. To party rather than study was, of course, her choice, but that doesn’t take into account the facts of Darci’s chaotic home life – her parents had divorced then remarried and divorced again in a short space of time, and various stepfathers had come in and out of her life. In her teens, Darci was essentially left alone to parent herself. You feel a stab of sadness when Potts talks to Darci’s mother, Virginia, who is now raising her daughter’s children, and finds her repeating the same mantra: “Darci made her own choices.”

By contrast, in the Potts household, Monica’s mother, Kathy, kept a close eye on her daughter, encouraging her to concentrate on her studies and finish school so she could get out and avoid the fate of so many Clinton women who got pregnant young, had few skills and married men for the perceived security it offered. This, essentially, is what happened to Darci: having flunked school, and given up on her college ambitions, she met George, who, like her, loved drugs and music. The pair fell into a relationship and Darci soon discovered she was pregnant, prompting her to settle down with a man she didn’t love, and who was prone to violence.

For much of the time, Potts’s writing maintains the dispassionate, fact-based tone of a reporter, though she can’t hide her disdain for the local Southern Baptist church that rains judgment down on young women for having sex outside marriage while viewing the conception of any child as part of God’s plan. For women, the message is clear: they must live under God’s authority and that of the men in their lives. As Potts observes: “Religious communities intend for women to pray for everything, not to have the power to make decisions for themselves.”

Elsewhere, in between relating Darci’s and her formative years and their subsequent reconnection, she delves into the impact of educated people like her leaving their home towns (it’s known as the “rural brain drain”), the decline of local industry, the broken health system, the lack of financial safety nets and the hidden trauma of families in which addiction, violence, sexual abuse and neglect are rife. “Places like my hometown,” writes Potts, “are still romanticised as the American heartland, but the reality of life drifts further from that idyll every year.” Potts offers no easy solutions but, in focusing on Clinton and the diverging fates of two best friends, she shows the cycle of despair blighting rural communities and the human cost of doing nothing.

• The Forgotten Girls by Monica Potts is published by Allen Lane (£20). To support the Guardian and Observer order your copy at guardianbookshop.com. Delivery charges may apply.