In The Man Who Mistook His Wife for a Hat, Oliver Sacks wrote about two autistic men, twins who had an extraordinary relationship with numbers. Sacks recalled that during one of his sessions a matchbox fell from a table, disgorging its contents, and the twins cried out “111”, the exact number of matches that lay on the floor. They later explained that they had not counted the matches, but could “see” how many there were. “The twins live exclusively in a thought-world of numbers,” Sacks concluded. “They have no interest in the stars shining, or the hearts of men.”

Daniel Tammet only mentions Sacks once in his new book, Nine Minds, but the writings of the British neurologist haunt these portraits of autistic people. Like those twins, Tammet is on the autistic spectrum, can perform absurdly complex calculations in his head, and is able to live in a “thought-world of numbers”, and yet he is also a writer whose qualities contradict the oft-made assumption that autism and empathy cannot coexist within the same mind. Part literary experiment, part work of activism, his book is an extended riposte to the assumptions in Sacks’s chapter; he celebrates the gifts and talents of autistic people, while exploring the richness of their desires and dreams.

The subjects of this book – which include a murder detective, a distinguished mathematician and a surgeon – all possess exceptional abilities: they can spot patterns, solve crimes and equations, and play creatively with language. And yet their lives are characterised by a great tension, a continual trade-off between gift and difficulty, lucidity and bewilderment, praise and alienation, acceptance and rejection.

Tammet tells the story of Vaughan, a brilliant hand and wrist surgeon who is widely admired for his skill, but who cannot remember faces and who struggles to make sense of what his wife is saying to him. He “had always been dexterous – he could pick up and repair a shattered vase, shard by shard, repair a hand, bone by bone. But the fragments of a mind – that was something else altogether. So hard to grasp and reassemble.”

Autism presents a particular challenge to our assumptions about language. Of Dan Aykroyd, the comic actor, Tammet writes: “He didn’t listen to his partners on stage as the others did; he fixed on specific words in a sketch, either written in advance or just that moment improvised, and on the flood of associations they set off inside him.” It is not that Aykroyd, nor any other of Tammet’s subjects, does not listen, but that they listen and perceive differently. This is a quietly radical revelation which speaks to the essence of the neurodiversity movement: if there is a deficit in society, then it does not lie in an autistic person’s differences of perception, but in neurotypical assumptions that govern human relations.

Tammet himself grew up before the neurodiversity movement gained steam, and at times his book reads like a long letter to his younger self, the boy who grew up surrounded by the language of disability not difference. Like a novelist, he enters his characters’ heads, often imagining them as children; he reconstructs dialogue and shifts time and place. Rarely does he quote directly from them, and instead of the ambiguity and open-endedness of their speech, we have Tammet’s own authorial voice, the “10th mind”, whose coating lies thick over these portraits, at times revealing, at others obscuring the inner lives of his subjects.

It is now almost 40 years since Sacks wrote about those twins; since then, many writers have sought to find their own language to describe autism and liberate it from the case study. Tammet’s work is part of this project, and yet I was sometimes unsettled by the framing of this book and its focus on plaudits and success: the emphasis on seeing what others don’t see and solving what others cannot solve. This is a useful corrective to the history of prejudice against autistic people, and the lack of appreciation of their talents, though a focus on exceptional brilliance seems a fragile foundation on which to build a plea for tolerance.

For all that his subjects are neurodiverse, their preoccupations and desires seem all too typically human: to be seen, loved and understood by others. I enjoyed most those chapters that paused to dwell on their relationships with friends, family, doctors and colleagues; the networks of care on which we all rely, autistic or not.

One of Tammet’s best portraits is that of Kana, a Japanese academic who specialises in loneliness among autistic adults. She herself feels this emotion acutely in cities, and ended up settling in Okinawa. As Tammet explains, the Okinawans have a culture of yuimaru, which translates as “looking out for one another”. It may be that there already exists a language that embraces autism and difference, we just need to open our ears to it.

• Nine Minds: Inner Lives on the Spectrum by Daniel Tammet is published by Wellcome Collection (£20). To support the Guardian and Observer order your copy at guardianbookshop.com. Delivery charges may apply.