If we accept Vladimir Nabokov's judgment that "a good reader, a major reader, an active and creative reader is a re-reader" and place it alongside Albert Camus's remark that "the whole of Kafka's art consists in compelling the reader to re-read him", we might conclude that Kafka's work is among the most valuable of literary treasures. This seems obvious to those who consider it, as I do, manically funny, desperately sad and endlessly rewarding; less so to those who find it baffling and inconclusive. Franz Kafka is one of the best writers for readers who love asking "What does it mean?", one of the worst for those who want that question answered.

Such is his stature, however, that like him or not, he can't be omitted from any discussion of the short story. And once his name is mentioned, the urge to explicate irresistibly follows – for, as Erich Heller wrote, he is "the creator of the most obscure lucidity in the history of literature, a phenomenon that, like a word one has on the tip of one's tongue, perpetually attracts and at the same time repels the search for what it is and means".

How does Kafka arrive at this junction of clarity and enveloping murk? A possible answer can be found in The Metamorphosis (1915), in one of literature's most famous opening lines:

When Gregor Samsa awoke one morning from troubled dreams, he found himself changed into a monstrous cockroach in his bed.

What is the significance of Gregor's transformation? The story, like most of Kafka's work, appears to be both allegorical and very real ("'What's the matter with me?' he thought. It was no dream."). Gregor's travails can be seen as operating on both a highly symbolic and a literal level – and this duality is fundamental. The greatness of The Metamorphosis comes not from the ingenuity with which its meaning is hidden, but from how its parallel meanings, however many there are, are as powerful and as affecting as one another. While asking ourselves what it is that a man turning into an insect might signify, it is equally important that we also read The Metamorphosis as a story about a man who has turned into an insect.

It was aged 29, during one night of non-stop writing in September 1912 that, in the words of the translator Michael Hofmann, "Kafka became Kafka". This creative frenzy produced The Judgment, a story in which a morning conversation ends with a father condemning his son to death by drowning. It contains much that would come to characterise Kafka's fiction: a sedulous, unadorned writing style; narrative momentum; sudden tonal shifts; the complex meshing of everyday reality and nightmare; cruel, arbitrary punishments; and a world that looks like the one inhabited by you and me, but in which the inner turmoil of the psyche is on the loose.

Kafka's ahistorical stories are modernist works, but his is a modernism broadcast from the cultural fringes. German in language and culture, Jewish by ethnicity, and living in Prague, the third city of the Austro-Hungarian Empire, Kafka was thrice isolated. Considered alongside the uniqueness of his literary vision and a complicated personal life, his lonely position makes it easier to understand the genesis of such powerful meditations on art and life as Josephine the Singer, or the Mouse Folk (his final story) and the surpassingly beautiful A Hunger Artist, the proof of which he began correcting on his deathbed. These profoundly lucid later works, written after Kafka had been diagnosed with tuberculosis and pensioned off from work, stand in contrast to the stark horror of earlier pieces such as The Judgment and In the Penal Colony (the latter so brutal it troubled its creator), but they are unmistakably different facets of the same artistic vision.

The greatest danger to the work is the glut of analysis it has spawned – what Milan Kundera derides as "Kafkology" – which persuades people that they know the stories without having read them, or that they have been "explained". With Kafka the individual response is everything, and this response, in accordance with Nabokov, alters in some way on each re-reading. The best advice I can give, then, is to toss this aside and engage with the stories right away. Whether it's for the first time or the 100th, the experience will be unique.

Next: MR James