Being John Burnside was a remorseless exercise in human resilience. Even after he left behind childhood deprivation and became a professor at an ancient university, there were always demons to be banished, myths to be busted and issues urgently to be addressed for Burnside, who has died aged 69 after a short illness. Writing was his best revenge, the means by which a sense of stability could be imposed on personal turbulence.

It was his abusive, alcoholic father George who taught him how to make a living by telling stories, though the stories his father told were more often than not lies. “No one,” recalled John in A Lie About My Father (2006), the first of three celebrated memoirs, “ever did find out where my father came from. He was really a nobody: a foundling, a throwaway. The lies he told were intended to conceal this fact, and they were so successful that I didn’t know, until after he died, that he’d been left on a doorstep in West Fife in the late spring of 1926, by person or persons unknown.”

John’s debut collection of poetry, The Hoop, appeared in 1988, and few were the following years when there was no new book from him. His poetry, like his prose, often drew on the past and his own experience, such as The Woman Taken in Adultery, from A Normal Skin (1997), and Memories of a Non-existent Childhood, from Still Life with Feeding Snake (2017): “For years I was lost in the details / heart like a flower, / tending towards the light, / the fog of the cursive, / the beauties of mistranslation.”

For his 2011 collection Black Cat Bone, he won the TS Eliot prize and the Forward prize; he was one of only four poets – the others being Ted Hughes, Sean O’Brien and Jason Allen-Paisant – to win both prizes for the same book. In 2000, The Asylum Dance won the Whitbread poetry award. A Lie About My Father was chosen as the Scottish Arts Council’s nonfiction book of the year and the Saltire Society Scottish book of the year.

Last year he was awarded the David Cohen prize for literature in recognition of his entire body of work. Previous winners include VS Naipaul, Muriel Spark, Doris Lessing and Seamus Heaney.

Secrets, passed from father to son, defined John’s youth and obsessed his adulthood. He was born in Dunfermline, but was raised in a prefab in Cowdenbeath. Once a thriving mining community, nicknamed “the Chicago of Fife”, by the time of John’s birth it had lost its raison d’etre and everyone who could escape did.

For the Burnsides the hoped-for nirvana was the new town of Corby, Northamptonshire, which attracted so many unemployed Scots to its steelworks it was known as Little Scotland. By then, however, as John recalled in his memoir, his father had begun to fall apart, physically and mentally; his much-loved mother, Theresa, did her best to keep up appearances, attending mass and reading Mills & Boons by the yard.

For his part, John took to smoking dope and playing “childish pranks” for which he was expelled from school. By 16 he had progressed to LSD, which he found more rewarding than the communion host. “Acid did what the host failed to do,” he wrote. “Acid was the only real sacrament to which I had access … Here I was, the boy who had seriously thought about a vocation. Now, though the source wasn’t quite what I’d expected, I had one.”

Of an autodidactic tendency, he read deeply and widely, claiming he was “a Seneca nut into my 20s”. He attended Cambridge College of Arts and Technology (“for something to do”), thereafter becoming a computer software engineer. He began to publish poetry in the 1980s, and became a full-time writer, moving back to Fife in 1996 after a long period in Surrey.

In 1997 came The Dumb House, the first of John’s eight novels. Fantastical, disturbing, chilling, it is full of startlingly arresting sentences that provoked critics such as Karl Miller to describe the author as an “extraordinarily good writer”. Rereading it recently, I caught echoes of Proust: “For a long time, I refused to speak – or so my Mother told me.”

For John, his formative years in Cowdenbeath and Corby were the ore from which he sought to extract diamonds. In Living Nowhere (2003), for example, he returned to the Corby of the 70s, where the hope of a better future for its transplanted inhabitants and their offspring is dashed by violence that hangs over the town like the ash and stench from the steelworks.

His taste was catholic, his enthusiasms boundless and his curiosity infectious. In his third memoir, I Put a Spell on You (2014), he describes incidents from his life and muses on popular music, classical literature, old and foreign movies, Diane Arbus and Mel Lyman, the American folk musician and film-maker who provided a link between Woody Guthrie, Bob Dylan and Andy Warhol.

The book’s title is taken from the much-covered song written by Screamin’ Jay Hawkins, another maker of his own myths. The version John first heard as a nine-year-old was by Nina Simone and whenever he heard her or anyone else sing he was transported back to Cowdenbeath. There, as a teenage barfly, he heard a girl called Annie sing it in a cafe, not long after which she was murdered.

Burly, bespectacled, latterly bearded, John taught for many years in the English department at St Andrews University, where he became professor in creative writing in 2009. Among his colleagues were Douglas Dunn, Robert Crawford, Kathleen Jamie and Don Paterson.

As my wife and I witnessed when we paid an annual visit to the university, John was an impassioned and eloquent educator and revered by his students. Not so long ago, he sent us a copy of his 2019 book The Music of Time: Poetry in the Twentieth Century, generously inscribed and thanking us for “the criticism and the encouragement over many years”. Not many writers are so forgiving of reviewers.

In 2018, writing in the London Review of Books, John told how he was suffering from sleep apnoea, “a condition where the patient stops breathing while asleep, then starts awake and desperately gasps in some air, before settling down again, all in a matter of seconds”. Two years later, in the midst of the Covid-19 pandemic, he suffered “huge heart failure”, as he described it, and was given a “do not resuscitate” order by doctors who feared he would not pull through. Somehow he did and continued to work at full pelt.

In Ruin, Blossom, his most recent collection of poetry, published earlier this year, John explored ageing, mortality and the parlous state of the environment. Having long since rejected organised religion, he labelled himself a “deep ecologist/anarchist”. He was particularly exercised about the building of a windfarm beside a nature reserve near his home a few miles south of St Andrews, not because it spoiled his view but because of the harm it would do to bats, birds and other wildlife.

He wrote regularly for publications including the New Statesman, the TLS, the New Yorker and the Guardian, and became a fellow of the Royal Society of Literature in 1998.

He is survived by his wife, Sarah (nee Dunsby), whom he married in 1996, their two children, Lucas and Gil, and their grandson, Apollo.

• John Burnside, writer, born 19 March 1955; died 29 May 2024