As per that ancient Chinese curse, SF and fantasy have been living through interesting times. This year attempts to rig the Hugo ballots by disaffected, right-wing, mostly American fans – known as the Sad Puppies and the Rabid Puppies – were heartily rebuffed by fandom more generally. The Puppies have not gone away, but as the year ends, it seems their influence is waning. More recently, the World Fantasy Awards decided to change their trophy from a bust of HP Lovecraft’s head to something else, as yet undecided, on the grounds that identifying the award so closely with a man who was in life so racist and antisemitic was inappropriate. Some on the right reacted angrily to this decision, as though Lovecraft himself were somehow being censored. He is not, of course: his books are as available, and as widely read, as ever.

What we are witnessing is a struggle for the soul of the genre. In the blue corner are those who look back nostalgically to a notional “golden age”: mostly white, male, American SF, with exciting adventures, accurate science, cutting-edge technology and a sense of wonder. In the red are those who see SF now as a global literature, hospitable to the alien and to otherness, an imaginative space in which old certainties are challenged, new stories get told, and where the pulp increasingly crosses over with the literary.

This battle will be won not in the newspapers, comments threads or posturings over awards. It will be won on the level of who is writing the best SF and fantasy. And the best writing this year came from all around the world, from women as much as men, and most of all from writers prepared to try to make it new. It’s not that SF is jettisoning its rich backlist, rather that the best writers are engaging with the traditions of genre in more thoughtful and open-minded ways.

A case in point is Kim Stanley Robinson’s Aurora (Orbit): a novel that works wonders with the venerable trope of the generation starship. Detailed, humane and with impressive cumulative narrative drive, Aurora asks profound questions about the viability and the ethics of interstellar colonisation. It gets my vote for the year’s best genre novel. Other standouts: Ann Leckie published Ancillary Mercy (Orbit), and so brought her Imperial Radch trilogy to a satisfying close. Alastair Reynolds’s Slow Bullets (Tachyon) is a treat: a punchy short novel set in the aftermath of an interstellar war. Neal Stephenson’s characteristically sprawling and readable Seveneves (Borough) starts, boldly enough, with the end of the world and picks up pace from there. Ian McDonald’s cut-throat colonised moon in Luna (Gollancz) is not the sort of place you’d want to live, but it certainly makes for a gripping narrative. Becky Chambers’s charming space opera The Long Way to a Small Angry Planet (Hodder & Stoughton), originally self‑published, got a big-publisher release this year. Liu Cixin’s impressive The Three Body Problem (Head of Zeus) won the 2015 Hugo award for best novel, the first time a Chinese writer has taken that prize.

There is more to SF than outer space. If you can read Sarah Pinborough’s YA fable The Death House (Gollancz) without crying, then you’re made of sterner stuff than me. A group of kids, all genetically “defective” in some way, are quarantined in the titular house; Pinborough’s future world is sketched in, to throw the emotional highs and lows of her doomed youngsters more effectively into relief, and the final sections are heart-wrenching. Anne Charnock’s Sleeping Embers of an Ordinary Mind (47North) is an expert braiding together of past, present and future that puts a 15th-century Italian female artist centre stage to say penetrating things about womanhood, creativity and history. James Bradley’s elegantly bleak vision of a climate-change future, Clade (e-penguin), made a bigger splash in its native Australia than over here, but deserves a wide readership: it’s urgent, powerful stuff.

Dave Hutchinson seems to have gone from unknown to the front rank of British SF in dazzlingly short order. Actually his is an overnight success story that took decades of hard graft – and Europe at Midnight is a wonderfully written novel, a worthy sequel to his extraordinary Europe in Autumn.

This was the year David Mitchell (below left), one of our best novelists, fully came out as an SF geek. Mitchell’s complex and stimulating novels have always been heavily dosed with genre; his latest work, Slade House (Sceptre), a horror fantasia set in the same imaginative universe as The Bone Clocks, developed out of a short story first published as 280 tweets. It’s perhaps not major Mitchell, but any novel from this writer is big news.

There was no new novel from China Miéville this year. Instead we had Three Moments of an Explosion (Macmillan), a collection of short fiction which ranks alongside his best work. That it is an assemblage of various pieces actually plays to Miéville’s strengths: the apprehension of social fragmentation and somatic alienation through an endlessly fertile, monstrously inventive imagination. This year also saw the publication of China Miéville: Critical Essays, edited by Caroline Edwards and Tony Venezia (Gylphi Limited), a sign that the academy is finally catching up with this major figure.

• Adam Roberts’s Bête is published by Gollancz. Browse all the critics’ choices and save up to 30% at bookshop.theguardian.com or call 0330 333 6846. 20p from each book you order until Christmas will be donated to the Guardian and Observer charity appeal 2015.

Best books of 2015

• Fiction of the year
• Crime and thrillers of the year
• Biographies and memoirs of the year
• Sports books of the year
• Children’s books of the year
• Nature books of the year
• Politics books of the year
• History books of the year
• Paperbacks of the year
• Music books of the year
• Food books of the year
• Drink books of the year
• Stocking-filler books of the year
• Vote now: what was your favourite book of the year?