Yes, it’s that time again: next Sunday the winner of this year’s Booker prize will be announced. Everyone has their favourites and it’s easy to carp about novels the judges missed, but the truth is we don’t know which ones they consider – the list of 150-odd titles is never made public and probably any number of subterranean factors determines its makeup; a few years ago I heard of at least one big-name British author, an obvious contender, who refused to be considered. And awards – now brands in themselves – seek to establish their own identities: was Barbara Kingsolver’s Demon Copperhead crossed off the list after winning the Pulitzer and Women’s prizes for fiction?

One of the judges, Shakespeare scholar James Shapiro, recently said he isn’t comfortable with historical fiction, which perhaps had something to do with the absence from the longlist of two of the year’s most feted novels, Zadie Smith’s The Fraud and Tom Crewe’s The New Life. True, one historical novel has made the shortlist: This Other Eden (Hutchinson Heinemann), by Pulitzer-winning US author Paul Harding – but that’s no surprise: the chair of judges, Esi Edugyan, says how good she thinks it is on the cover of the book itself, published last February.

Drawing on the true story of the forced eviction of a mixed-race island community in New England before the first world war, it may have Edugyan’s vote but I think it’s a novel that’s hard to love. We follow various descendants of a freed slave as a well-meaning missionary, privately racist, attempts to spare one of them – a gifted young painter who happens to pass as white – by having him moved to a friend’s estate on shore.

At the core of what ensues is his doomed romance with his landlord’s Irish maid. But Harding relies on laboriously laid slabs of narration to generate dramatic irony as we see events from different points of view – the painter’s, the maid’s, then his family’s, confronted with her pregnancy as she visits the island in search of the painter’s past.

In his previous novel, Enon, Harding found a measure of comedy even in the plight of a father mourning the death of his teenage daughter in a road accident. Nothing to laugh about in This Other Eden, a mark of Harding’s sense of responsibility to his painful material. For sure, he wrings pathos from the injustice and horror from the climactic violence but the novel never justifies its own interest in the spectacle.

I have similar unease about Paul Lynch’s Prophet Song (Oneworld), set in a contemporary Ireland gradually riven by war between a totalitarian regime and insurgent rebels. It follows a Dublin mum of four after the arbitrary detention of her trade unionist husband. The focus is on her trials keeping the breakfast-to-bedtime show on the road amid a descent into hell, from abductions to air raids. That focus gives the novel its convincing frisson – as bombing intensifies, the protagonist’s father rings her up complaining about his neighbour’s ivy – but it also locates it firmly as thought experiment. It’s jarringly apolitical – there’s no obvious history to what’s going on – and ultimately there’s something almost obscenely decadent to its invitation to sympathise with sea-crossing refugees by recasting them as middle-class Europeans. When Eilish thinks it’s “as though some filmed transparency of a foreign war has been placed upon an image of the city”, the line catches exactly what Lynch is up to.

Still, its explosive set pieces have a visceral spell undimmed by rereading, and any sense of unease it fosters may count in its favour – whatever else it is, Prophet Song is a novel to argue about.

I preferred quieter books on the list. Kudos to the judges for spotlighting If I Survive You (Fourth Estate), a thrilling debut by US writer Jonathan Escoffery (a former student of Percival Everett, shortlisted last year for The Trees) that starts as an exploration of second-generation American identity by way of a Jamaican-descended literature graduate, Trelawny, whose morale – already battered by being second-best, at least in his father’s eyes, to his tree surgeon brother Delano – takes another hit by his graduating post-recession. He finds himself doing crazy things for money, not least being paid by a woman who puts out a classifieds ad looking for someone to hit her.

With much tenderness in the gulf between father and son, this would be a worthy winner. It’s funny, sad, always toying with assumptions about race. Don’t be misled by the opening sequence, which feels like an exercise in voice, as the second-person narration explores the contortions of internalised racism. In later sections Escoffery shines through as a first-rate scene-maker with a wicked knack for jeopardy: witness Delano’s catastrophically ill-conceived bid to retrieve an impounded cherrypicker so he can take on a job. In the title story, Trelawny winds up part of a rich white couple’s twisted sex game (and yes, this is a linked story collection, openly advertised as such on the dust jacket, a first for a prize now awarded to “longform fiction”).

Another debut that is a revelation is Western Lane (Picador) by Chetna Maroo, the only British writer here (hard not to note that she got her break from magazines in the US and Ireland). Families were clearly a hot topic for this year’s judges, but the simplicity of this book stands apart. It is narrated by 11-year-old Gopi, youngest of three daughters in an Indian family living in south London, whose father schools her in squash at a local leisure centre in the year after their mother’s death. The story evolves as she practises to compete in a tournament: the power of the novel lying in what goes unsaid between Gopi and Pa, between her and an older boy she hits with at the leisure centre, between the sisters, between her and the childless aunt and uncle in Edinburgh where she’s expected to move.

This is a beautiful tale of family – and sport. A moving yet non-maudlin novel about grief, it’s told in plain language electric with feeling, nicely catching the narrator’s incomplete understanding of her situation without making her a figure of irony. A real feat: if the metric is the ratio of number of words to emotion generated, Western Lane wins.

Paul Murray’s mighty The Bee Sting (Hamish Hamilton) is a book nearly big enough to contain the others on the shortlist, and it features many of their themes, from filial misunderstanding to civilisational collapse – or its looming shadow, seen in the ultimately catastrophic prepping of its beleaguered patriarch, a car salesman sunk by the recession, wed to the class-climbing daughter of a bare-knuckle boxer. A 650-page soap opera with a clockwork plot whose delicate mechanism you can’t hear tick: if it wins the Booker there will be a lot of delighted readers finding it under the Christmas tree.

It’s so obviously the best novel here – but will the judges talk themselves out of it? A sceptical 3,000-word review in the London Review of Books shows what might happen if you’re encouraged to think about it for too long; and remember, too, the Booker doesn’t reward the best novel of the year – not exactly – it’s more the book that survives a process that entails being read and discussed again and again over the better part of nine months.

It’s because of this that I wonder if Prophet Song will get the nod – or Sarah Bernstein’s Study for Obedience (Granta). A Canadian living in Scotland, listed this year as one of Granta’s best young British novelists, she is also winner of Canada’s prestigious Scotiabank Giller prize (twice won by Edugyan). Study for Obedience has nagged me since I read it in March. I didn’t like it then and I don’t love it now but in a list of novels about family it stands apart – an almost spectrally vague existential brainteaser soaked in 20th-century bloodshed. I can’t tell you what the novel is about, exactly, and that’s why I think it bears scrutiny.

The narrator is an unnamed woman who moves to an unspecified country to take up her creepy elder brother’s affairs as a housekeeper, where her duties extend to soaping his back, and she’s permitted to watch TV only by lipreading (he says subtitles affect the play of light in the room). Strange encounters with locals never quite bloom into confrontation. Instead of drama we have discomfort and dread: between the lines lies the genocide and exile of the protagonist’s ancestors.

Trapping us in an isolated consciousness, the winding narration is full of comma-spliced caveats and double negatives that make it purposely awkward to place.

How to measure this against, say, the arrowlike flight of Maroo’s sentences aiming at the heart? I don’t envy the judges. Where they have by and large selected novels that show an uncomplicated faith in the pleasure of storytelling, Bernstein’s is the book that makes narrative a problem – the reader’s problem, some might say. But it just might be declared best on 26 November by judges who can’t be expected to spend a year agreeing with one another about how great The Bee Sting is.

• To buy any of these books for a special price and to support the Guardian and Observer click on the links above or order your copy at guardianbookshop.com. Delivery charges may apply