I know too much about Joan Didion. I’ve seen her image on tote bags and Celine ads; I’ve scrolled past her beautiful sulking face on countless Instagram feeds; I’ve streamed multiple documentaries about her. You’ll notice that I’m not describing her work; I’m describing her fame: the Didion spectacle. I’m impatient with this fame of hers. I believe she is famous for the wrong reasons; in other words, that she is loved wrongly and maybe too widely. I want her to be loved only the way I love her: as the author of a few specific pieces of writing.

I’m not alone in this feeling. My kid texted me the day Didion died: “I loved her but Twitter today is going to be unbearable.”

So I came with some trepidation to The Uptown Local. Some reviewers have complained that the book – an account of Cory Leadbeater’s time working as Didion’s assistant – is a little too light on the Didion and a little too heavy on Leadbeater’s own experiences of joy and death. And it’s true: there’s very little Didion dishing that happens in this book. But as a person who would like to know a lot less about Didion, it turns out I’m a good reader for Leadbeater’s memoir.

When Didion deus ex machinas into his life in 2013, Leadbeater is a young grad student at Columbia University, wildly out of his depth, desperate to write something great, and above all trying to distance himself from his abusive father and their working-class New Jersey family. Leadbeater answers an ad from a writer looking for an assistant without knowing who that writer is. Once he gets the job, he is swept up into the loftiest heights of New York literary culture, an ingenue spirited away to the brainiest ball.

As Leadbeater spends his golden working hours at Didion’s apartment each day – and eventually moves in – another story is unfolding back in New Jersey: his violent, self-aggrandising dad is indicted for real estate fraud. Unable to reconcile these contradictions, Leadbeater begins to fall apart, working on a failed novel, drinking too much and ultimately growing suicidal. This falling apart is the main thread of the book, with the details and texture of his friendship with Didion like a fabulous background heartbeat.

This is ultimately a book about distances. The distance between Leadbeater’s nascent, frustrated literary efforts and the great body of work that lies behind Didion; the distance between his violent, scrappy childhood and the rarefied atmosphere in which he finds himself, struggling to wield a table knife and/or a sentence properly; the distance between youth and age.

Leadbeater’s difficulty in maintaining his equilibrium among all these different ways of being becomes the central theme of the book. Strangely, fascinatingly, he achieves an epiphany on this score as he reflects on the spectacle of “Didion the myth”. He writes, echoing my frustrations about her image, “I had for so long struggled to understand why everyone got her wrong. She was for some the genius waif leaning against her Stingray; for others she was the lonely widow, padding aimlessly around her apartment, bereft forever without her husband and daughter; for others still she could be the political journalist and assassin, the California Cassandra, or else the vulnerable woman perpetually in bed with a headache.”

Leadbeater goes on to question the primacy of any particular image of Didion – she was all of those things and, he writes: “She did not try to reduce life down to a more manageable size in order to understand it; instead she endeavored to create a consciousness as large, varied, complex and contradictory as life itself. She rejected orthodoxy to better see the real.”

I myself have been guilty of turning Didion into a myth, even as I complain about others doing it. When we’re in traffic we forget that we are traffic. I forget that I am one of the makers of the Didion spectacle, fashioning my own version out of my very specific passion for very specific works of hers. Leadbeater makes us see there was so much more to her (to all of us) than just one thing. Strength, Leadbeater learns from the woman herself, means not cracking up when confronted with life’s irreconcilable distances, but somehow living with them.

• The Uptown Local: Joy, Death, and Joan Didion by Cory Leadbeater is published by Fleet (£22). To support the Guardian and Observer, order your copy at guardianbookshop.com. Delivery charges may apply.