By 1994, I’d published seven crime novels and I’d been writing full-time for three years. The first two years, my writing had been a tax loss. I’d only stayed afloat because of the redundancy money I got when I quit my job. But things were slowly improving thanks to my agent’s assiduous sales of foreign rights and the occasional library event. I had two series characters – Scottish journalist Lindsay Gordon and Manchester PI Kate Brannigan, but although the settings and preoccupations of the books were contemporary and feminist, structurally they were well within the traditions of the genre.

I was living on the edge of the West Pennine moors, in a quiet suburban street that did not feel like home to me. For most of the time I was writing The Mermaids Singing, I was desperately trying to sell the modern bungalow that my ex had talked me into buying. One sunny morning, I was driving home down the M6, half-listening to Woman’s Hour, enjoying the rolling curves of the Howgill Fells, when a plot popped fully-formed into my head. I knew what the story was, who the killer was, who the victims were, how the events unfolded and who the protagonists were. It was profoundly shocking.

Books don’t start like that for me. They begin with a small thing. An interesting nugget of information. An anecdote that makes me wonder: “Yes, but what if …?” A throwaway line in an article or a radio programme. Eventually, they either start to make narrative sense, or they fade into oblivion. It can take years from the first glimmer of an idea to a finished book. I think the record is 12 years of tinkering…

But The Mermaids Singing was essentially all there in the moment. I was so taken aback, I pulled over on to the hard shoulder to write it down in case it had evaporated by the time I got home. It felt like a narrative emergency.

Hard to imagine now, but in 1994, nobody in the UK was writing serial killer novels with psychological profilers. My own interest had been piqued by Thomas Harris’s first Hannibal Lecter novels, and I’d read the accounts of FBI profilers Robert Ressler and John Douglas. In that sense, I’d ploughed the field in preparation for the seed of inspiration.

The first stumbling block was that the UK didn’t have psychological profilers. When the police needed insight, they turned to practicing clinical psychologists and sought their help. And I didn’t know anybody who did that sort of work. Then the writer’s best friend, serendipity, stepped in. On the local news, I saw an interview with the very man. Long story short, he agreed to talk to me and my character Tony Hill’s method was born.

The next problem was finding an illustrated guide to medieval torture. This was before the internet had much content, and I got little joy from freaked-out librarians and booksellers… Then serendipity struck again. I was on holiday in Tuscany and I saw advertisements all over Florence for the new Museo Criminologico di San Gimignano. Sounded interesting so off I went. It was a museum of torture instruments. I think I was the only visitor cheerily taking photographs of the horrors within. I was ready to roll. I realised I needed a split narrative – the voice of the killer, whose timeframe started ahead of the main multi-viewpoint third person narrative. It was structurally far more complicated than anything I’d attempted before. It was also thematically and atmospherically very different from my previous work. I had so little faith in what I was doing that I sent the first hundred pages to my agent. The relief was overwhelming when she got back to me and said simply, “Just get on with it, darling. It’s marvellous.”

I wrote it very quickly. The first draft took about four months. My brilliant editor, Julia Wisdom, worked hard with me to polish it and make it as good as I could get it, though thankfully there wasn’t much structural rewriting required. The plan was to publish it in November 1995. But because of the time frame of submissions for the Crime Writers’ Association Dagger awards, we had to send proofs rather than finished copies to the judges.

And that averted certain disaster. In one of those periodic publishing nightmares, the first edition of The Mermaids Singing was printed without the last four pages. It made it to the bookshops before anyone noticed and had to be recalled. If that had been the version the judges read, I doubt very much whether I would have won the Gold Dagger.

Winning the Dagger, at that time the only crime writing prize of any significance in the UK, changed my professional life. More bookshops stocked more of my work. Foreign markets opened up. Ultimately, it led to the TV series, Wire in the Blood, which had global success and still airs on streaming networks. More importantly, it brought more readers to my work.

The Mermaids Singing became a touchstone for me. It was so difficult to find the narrative voices for the book, but in the end, the struggle proved worth it. Now, whenever I sit down to a new novel and I worry about finding my way through the narrative, I remind myself of the huge leap of faith I took with The Mermaids Singing and that gives me the confidence to forge onwards. Deep down, I know I’ll find a way to tell whatever story is clamouring in my head.

• Christmas Is Murder, a short story collection by Val McDermid, is published by Sphere this week.