“Life is short and the world / is at least half terrible,” observes the poet Maggie Smith. But which are the books to reach for when the terrible half is in the ascendant? I’ve come to treasure a particular category I’d define as the literary comfort novel: elegant and beautifully written stories that renew our faith in humanity, that leave us better than they found us, that work – and thus expand – the muscle of the heart. Lately, I’ve come to realise that I want to read one story: despite everything, it is going to be OK.

That “despite” is key. Solace is distinct from escape, and I’d define a literary comfort novel as distinct from pure escapism. I don’t want the narcotic and temporary palliative of the soap opera – without judgment, I should add – but, perhaps because I have been aged or wounded out of fairy stories, I want a more convincing consolation, grounded in real life. The best books rebuild our strength so we can face reality, and maybe even fight to change it. I don’t want briefly to suspend my disbelief. Instead, I long to believe again.

Nick Hornby looks for “absorption, the possibility of jokes or wit, a writer who understands that their job is to entertain the reader. Optimism is nice too. If literature continues to paint itself into the miserable and difficult corner then it will die, and yield the ground it should be occupying to other forms of entertainment. So I am left only with Dickens, Austen, Wodehouse, Wells, Waugh, Green, Vonnegut, Strout, Patchett, Tyler – and about 500 other authors who wish to provide pleasure to the reader.”

Probably there will always remain a snobbery about the optimistic; certainly there endures the belief that optimism is inherently naive, and that a novel can only have literary value if it opens up or illuminates yet another chamber of horrors in the catalogue of human misery. A surprising number of readers believe a happy ending should mean automatic disqualification from any serious literary award. Good luck to them: I wish them joy in their wallowing. In my turn, I’ve come to believe the opposite. To reach only for novels that reaffirm our darkest fears is merely to make an escape of a different sort, not the escapism of brooding heroes and wedding finales, but the security blanket of an equally foregone conclusion: the safety of imagining the worst. I would argue that to live only in that place is simply cowardice in better camouflage. The truth is that it’s far riskier to remain in uncertainty. Far braver, far more radical to keep hoping.

Here, then, is my ultimate reading list for soul sustenance. Each is a guaranteed source of humour, compassion and insight; all are by brilliant writers whose sentence-by-sentence work will itself bring pleasure. If these are the only novels you read on loop for the rest of your life, I reckon you’ll be all right.

If there’s a more triumphant character in contemporary literature than Audra Cavanaugh, I don’t know her. In Katherine Heiny’s masterpiece, Standard Deviation, Audra is eccentric and inexhaustible, tender-hearted and spectacularly tin-eared, and somehow essentially unknowable to her older husband Graham, despite the fact that she would tell absolutely anything to anyone, and frequently does. Heiny captures the good-faith muddle of a real marriage under strain; a couple absolutely united in their devotion to their 10-year-old son but in little else, it sometimes seems, and it becomes increasingly obvious that Graham is not as straightforward and dependable a character as he himself insists. Here, fidelity and even honesty might be overrated – love, a sense of shared purpose and a robust sense of humour win the day.

Loving deception is also at the centre of Family Happiness by Laurie Colwin: another messy marriage, another sublime New York novel. Polly is the perfect daughter of the perfect Solo-Millers, for whom visits to her artist lover are enabled by the fact that no one in the family takes the remotest interest in her job, though they’d be alarmed if she forgot the olives for lunch. No tidy endings here either, just the bliss of a fantastic and underrated writer.

Speaking of underrated, more people in the UK should be reading Kevin Wilson. Nothing to See Here is possibly the ultimate literary comfort read. Lillian’s old school pal Madison calls for Lillian’s help (not the first time that impoverished, vulnerable Lillian is called in to cover for her rich best friend). Madison needs someone to mind the 10-year-old twins from her husband’s first marriage; the minor catch is that Bessie and Roland burst into spontaneous flames whenever they’re upset. Lillian is a dropout and a cynic, wickedly smart and relentlessly hilarious, while her growing connection with the two children is nothing short of transcendent. Most people are horrible and life generally sucks, suggests this novel, and yet. And yet. In unlikely places, humanity flickers and catches alight. It will all – somehow – turn out OK.

The novels that bring most comfort to writer Sathnam Sanghera all share landlocked settings, as a reader “both from the Midlands and with heritage from the Punjab”. He’s comforted when the world of the novel feels safe, and cites fellow Black Country writer Jonathan Coe as his ultimate comfort read because “there’s so much warmth to his characters”. Also Possession by AS Byatt, which is “a book about books, and therefore safety within safety”.

For warmth of characters, together with exquisite compassion and humour, James McBride’s The Heaven and Earth Grocery Store is unsurpassed. In the early decades of the 20th century, the residents of Pennsylvania’s Chicken Hill are not exactly living the American dream; African Americans and Jewish immigrants together contend, in their marginalised community, with racism, poverty and disability. They are challenged over and over, but this is an epic novel suffused with hope for humanity, and one which reminds us of the power of community, and love.

No list would be complete without an Austen, and my top nomination would always be Persuasion. Her most mature and refined novel, it is still the closest to an escapist fairytale, the Cinderella story of sweet, longsuffering Anne Elliot, burdened with and neglected by her appalling father and sisters. Austen leads us inexorably towards the proposal, the wedding, the man of good fortune, but along the way Anne suffers, and Austen’s clear eyes and sharp nib make the predictable miraculous. I challenge you not to sob when you read Frederick’s letter.

Everyone by now has read the wonderful Lessons in Chemistry by Bonnie Garmus: funny, poignant, unbelievably satisfying. I loved it – and in a similar vein, have you also read Where’d You Go, Bernadette by Maria Semple? The two have in common a gifted woman resisting the role in which she has been cast; losing and then triumphantly finding herself.

Rónán Hession’s Panenka breaks my rule that it must all turn out all right – but it’s a worthy exception. In a substantial sense it will definitely not be OK; it rarely is when the protagonist begins by mentioning that he suffers from the sort of headaches that deliver nights alone curled in agony on the bathroom floor. Yet with Hession’s tender treatment it remains a story of hope, and redemption. Recently reunited with his adult daughter and seven-year-old grandson, Joseph is a retired footballer living in a run-down town, and has for years been held single-handedly responsible for the declining fate of the beloved local team (and thus the town itself), after one misjudged penalty kick. It’s full of warmth and forgiveness, of reconnections and second chances, and what it is, in the end, that really matters.

Margaret Drabble’s The Millstone is a delight because nothing in Rosamund’s life should go as well as it does; one keeps expecting her to succumb to the cliches of 1960s unmarried motherhood and yet on she goes, skirting disaster, happy despite what anyone believes she ought to feel about her predicament. In short, she isn’t punished, either by her generation or by the imagination that created her. Ah, yes. There’s another condition I’ve laid down for my own favourites – writers who treat their characters with respect. I can no longer stomach watching novelists pull the wings off the flies they have written into being.

Tom Lake by Ann Patchett takes optimism almost as its guiding principle – broken hearts heal; lost young people eventually find their way into contented adulthood. It is a sublimely reassuring novel because of its structure: most (but not all) of the novel’s suffering is contained within a long-ago episode, recounted by a narrator who is now a happily married mother in her 50s, picking ripe fruit in the sunshine. If to some that lack of peril makes it saccharine, well, tough cheese. As Ann Patchett herself has said in these pages, “I am a glass-half-full, can-do kind of gal. It’s just the salt in my brain. So, people give me grief about being too hopeful or too cheerful or too interested in family – it doesn’t matter. I’m not writing all the novels.” Readers are free to give ourselves over to the journey, for we have already taken possession of a happy ending.

For the novelist Naomi Alderman a literary comfort read should be “a pep talk in the corner of the boxing ring of life”, and her own favourites are Susanna Clarke’s magisterial Piranesi, “which is really a book about surviving a long illness, but somehow also about how it’s OK to be changed by life, by hard times, by the world; that being changed can be a power as well as a wound”. And George Saunders’s Lincoln in the Bardo, “about a man facing the worst possible thing, the death of his child, and having to haul himself together to save his country. It’s a book that’s rich and tender, and so full of clear-eyed love for humanity – the belief that, with all our rough edges and unsavouriness, we are still worth saving.”

For that, after all, is the heart of what it is to be consoled. Holding fast to our faith that we are worthy; maintaining our determination to save ourselves, and one another. A truly successful comfort novel should leave us a small, smooth pebble of reassurance that we can close into our palm and carry with us, back into the real world, back into the fray.

• Welcome to Glorious Tuga by Francesca Segal is published by Chatto & Windus. To support the Guardian and Observer, order your copy at guardianbookshop.com. Delivery charges may apply.

The ultimate literary comfort reading list

Nothing to See Here by Kevin Wilson
The Heaven and Earth Grocery Store by James McBride
The Millstone by Margaret Drabble
Juliet, Naked by Nick Hornby
I Capture the Castle by Dodie Smith
Panenka by Rónán Hession
Standard Deviation by Katherine Heiny
Persuasion by Jane Austen
Tom Lake by Ann Patchett
Where’d You Go, Bernadette by Maria Semple
Lessons in Chemistry by Bonnie Garmus
Family Happiness by Laurie Colwin
Tomorrow and Tomorrow and Tomorrow by Gabrielle Zevin
Piranesi by Susanna Clarke
Lincoln in the Bardo by George Saunders