It was snowing and the heating had broken the day I visited the Mass Observation Archive in Brighton. I sat in my coat, woolly hat and fingerless gloves, my breath clouding the air. Before me were nearly two hundred anonymous letters written in 2007, most by hand, reflecting on the “ups and downs” of friendship.

Mass Observation is a treasure trove for historians like me. Since 1939, it has sent out loose questionnaires called “directives” to its pool of volunteer writers across the UK, who respond by describing their daily lives, opinions and feelings. Most who replied to the directive on friendship were women over 60. As the hours passed, my fingers grew numb but I didn’t care. It was so absorbing to read their intimate accounts of the pleasures and the usually unspoken difficulties of friendship.

There were terrible betrayals in those letters: deceptions, infidelities, backstabbings. But far more common was having a friendship that had become “difficult”. Once enthusiastic friends had now become relentless complainers; once supportive ones, prickly and overcritical. There were friends who always cancelled at the last minute, or only ever called to talk about themselves, or who subtly undermined the writers, leaving them “drained”. It was all painfully recognisable: I have had these friends. More to the point, I have certainly been this friend, too.

Each of these writers found themselves with a dilemma: should they end things or keep going? Though they were anxious about being disloyal or causing hurt, what seemed to worry them most was the idea of stringing the friendship out unnecessarily. Most chastised themselves for not “being braver” and making a clean break. They knew there was no legal or even social reason compelling them to keep up their friendships – unlike family or marriage, friendship is an entirely elective relationship with no contractual ties. They described irritated partners, who could not comprehend why they hadn’t cut the person off. They suspected younger readers would be more assertive: “I know what you’re thinking,” wrote one 72-year-old, who had taken to dodging the calls of an endlessly demanding friend, “I must be barmy. I plead guilty!” As a historian of emotions I am interested in how people wrestle with the gap between how they really feel and behave, and how they think they ought to. I want to know where the expectations they are beating themselves up with come from – and whether they still hold true.

People have always recognised that friendships can sometimes become strained. Cicero suggested letting go gently, slowly lessening the intensity of the bond; though he also suggested that needing to end a friendship might be the result of choosing badly in the first place – rather than that life’s twists and turns might change two people and their friendship in unexpected ways. But the idea that we “ought” to end a difficult friendship gathered real momentum in the 1980s, hand in hand with a new archetype: the toxic friend. The first mention I have come across of “toxic friends” appears in the psychologist Joel D Block’s 1980 book Friendship, in which he says: “Most of us bring both nourishing and toxic qualities to our friendships,” and though we might want to minimise the more negative aspects, “perfection in this task is unlikely”.

This rather forgiving and nuanced approach was quickly eclipsed as the market for self-help books exploded and the idea of the “toxic” friend caught people’s imaginations. “A toxic friend is one who undermines our confidence, saps our strength, feels happy when we fail, criticises our progress and dogs our steps with prophecies of gloom,” declared the 1993 guidebook How to Survive Practically Anything. A 1996 headline in Teen magazine asked: “Toxic friends: are they poisoning your life?” and three years later, Florence Isaacs’s book Toxic Friends/True Friends terrified readers by claiming the former could threaten your health, happiness, family life and career. The language was vivid and dramatic and the solution seemed obvious: what else do you do with a poison but cut it out?

The trend took off partly because it channelled a new spirit in which self-actualisation and self-improvement were king, and partly because, as sociologist Eva Illouz argues in her book The End of Love, “choice – sexual, consumer or emotional – is the chief trope under which the self and the will in liberal polities are organised”. She argues that we have learned to think of ourselves as managing our emotional lives and relationships in very intentional ways. Failing to act decisively, by not exactly ending a friendship but not exactly embracing it either, can, in this context, be seen as a sign of passivity, self-sabotage and a failure of will.

In our own age, online talk of “toxic friends” has grown even louder. Self-proclaimed social media experts explain the warning signs, while strangers exhort one another to ditch the friends that are causing such misery. As the sociologists Kinneret Lahad and Jenny van Hoof write, having a “toxic friend” is framed as a victimhood, while dumping them is “promoted as a courageous and healthy action … a desired form of ‘self-care’”. In the letters I read in the Mass Observation Archive, I saw what kinds of feelings might be obscured by this shrill, uncompromising vision. Were the writers really cowardly, avoidant or irrational, as they feared? Or was there something else that compelled them to keep their connection with their “difficult” friend alive? In their letters, I read how painful it is to contemplate ending a friendship. I saw the fear of hurting someone we have been close to, the ways writers had tried to understand the reasons for their friends’ changed behaviour, and the flicker of hope that things might change again: “I love her,” said one writer, trying to explain why she kept the friend around, “and I remember when she wasn’t such a nuisance.” Their experience of the friendship was messy, ambivalent, even tortured – a far cry from the idealised friendship imagined on social media, where even the slightest most subjective flaw or transgression can lead to the judgment that a friend is “toxic”.

But if a friend is touching on your sore points, and you on theirs, there are other ways to manage the strain. Though it might be quite intimidating for many of us, there’s the possibility of trying to talk it out. I have a friend who, after a terrible, guilt-inducing experience in her 20s with a friend she never spoke to again, has taught herself to be courageous, talking to people whenever things become strained. In one case, a friend she was trying to end things with ultimately became closer “because we have been through something very honest and raw together, and that friend now trusts me to say things that no other friend has ever told her before”. She believes that learning how to have these difficult conversations – and knowing that they can be survived – has been transformative, partly because it is easier to strike up friendships when you know what to do in the event that things go awry.

If the idea of such a conversation fills you with dread, it is also possible to step back gently, without feeling you have to create the kind of dramatic ending the “toxic friend” framework demands. Communications expert Emily Langan points out that, because we live in a culture that glamorises a particular kind of close and lifelong friendship, we can be seduced into imagining our friendships should be sustained at this pitch for ever. But there are many different types of friendships, from long-standing ones that carry on at a low simmer, to quick-boil connections, or more pragmatic, neighbourly relationships. In the same way, a single friendship may cycle through different sorts of intimacy.

And we can always, as many of the people I encountered in the Mass Observation Archive were doing, exercise patience, in the hope that the friend in question might recover from the difficult phase in their life, or that whatever distressing feelings we are struggling with ourselves might settle down. As these writers recognised, in a world that encourages us to swiftly end friendships that have been diagnosed as toxic, slower approaches requiring us to sit with ambivalent feelings can seem counterintuitive. Yet, ultimately, the more forgiving, flexible approach they teach can help us keep hold of the friends who are witnesses to our histories, as we are to theirs.

Tiffany Watt Smith is the author of Bad Friend: A Century of Revolutionary Friendships (Faber).

Further reading

The Virago Book of Friendship edited by Rachel Cooke (Virago, £18.99)

The End of Love: A Sociology of Negative Relations by Eva Illouz (Polity, £14.99)

Friendaholic: Confessions of a Friendship Addict by Elizabeth Day (4th Estate, £10.99)