Jo Ann Beard, 68, is an autobiographical essayist whose most famous collection, The Boys of My Youth, has become a staple of creative writing courses (Beard herself teaches the subject at Sarah Lawrence College in New York). At the time of its publication – 1998 – interest centred on the fact that one of the stories recounted Beard’s near-miss involvement in a workplace shooting. Since then, the collection has gained a cult following for its blurring of nonfiction and fiction and for its honest, intensely intimate portrayal of youth and adolescence. The Collected Works of Jo Ann Beard is now being published in the UK: Serpent’s Tail has brought together The Boys of My Youth and a more recent collection, Festival Days, in one book (Beard also published In Zanesville, a novel, in 2011). It is also issuing Cheri, one of the longer stories from Festival Days, as a novella. Cheri is the story of Cheri Tremble, a woman who, after prolonged suffering with cancer, underwent euthanasia. Beard lives in upstate New York with her partner, the writer Scott Spencer, and is the recipient of a host of prizes and awards including a Guggenheim fellowship.

Why do you think The Boys of My Youth gained such an extraordinary following, particularly among writers? It was a minor hit at the time of publication but is now viewed as a classic of creative nonfiction.
It’s unusual for a collection of essays by an unknown writer to reach a large audience, so I was gratified that people had the reaction they did. It’s the story of a middle child in a middle-class family in the midwest, and was only ambitious in its writing style and blending of genres. For some reason, it connected people to their own pasts and their own stories. And, as a memoir told in essays that read like fiction, perhaps burgeoning artists were attracted to the idea that their writing didn’t have to fit into one category and didn’t have to obey certain rules.

Two of the essays in Festival Days – Werner and Cheri – feel like they are particularly engaged with one of the central questions in your work: what is real and what is imagined. Both stories were the fruit of extensive interviews with those involved in the story – Werner Hoeflich, who almost died in a New York fire, and the family of Cheri Tremble, but they also draw deeply on the tools of fiction: you imagine your way into their lives.
In a way, I borrowed the stories and then tried to infuse them with metaphorical meaning. I wanted to experience, as a writer, what happens when you get all the way to your moment of death and then don’t die. Or in the case of Cheri, what happens on the march towards, and through, your own certain death.

Did you worry about how those involved would react to the stories?
I did worry about the response of the subject and their loved ones. It made me so panicky that I had to let go and just write from my own understanding and point of view. And I do want to state that nothing in writing is ever particularly enjoyable to me, except maybe at the very end of a piece when you can type: “The End”. Then for about 20 minutes, I feel good.

Are you writing now?
No, not for a while. I’ve always found it hard to focus, hard to get to that deep place you need in order for the writing to be worthwhile. I’m 68 now and I don’t want to multitask in the way that I used to. If you’re writing books, or essays, they feel so imperative compared with everything else in your life – and yet I am a teacher at a college in New York and that work is very demanding in a different way. As in, I get paid to do it.

Do you feel at all responsible for the popularity of autofiction?
No, there’s always been autofiction, but it was called something different. It was called fiction.

What appeals to writers about this form particularly?
It might have something to do with the fact that we are all – as artists, at least – plumbing our own experiences, our own depths, through the work. That necessary dedication to looking inward is what gives the act of looking outward more resonance. If we use ourselves as the template, we understand better what motivates others.

Which books are on your bedside table?
I never like to say because they never sound weighty enough. But here in my studio are two books that are helping me engage with my own writing mind: Holler Rat by Anya Liftig and Thin Skin, a collection of essays by Jenn Shapland.

How do you organise books on your shelves?
I flirt with a certain kind of OCD that causes me to really love things like maps and lining things up on the edges of desks. So I’m strictly alphabetical order. And nothing, believe me, is out of place.

Is there a classic novel you’ve recently read for the first time?
Yes, during the pandemic, and then twice more since then, I read Pale Horse, Pale Rider by Katherine Anne Porter. It’s the kind of book that you can finish over the course of two baths, and it’s instructive and beautiful and terrifying.

And what book would you give to a 12-year-old?
Another horse book: My Friend Flicka by Mary O’Hara. Also instructive and beautiful and terrifying, in its own, different way.

Are you excited by your students?
I’m very excited by literature, and so are my students, because they intend to make their own. Introducing them to work they haven’t seen before, or to new interpretations of work they already love, is the very best part of teaching. We move from that into exploring their own ideas and stories through language and image. If we’re doing it right, then each of us – as writers and as readers – ends up enlightened. To me, there’s no better way to spend a life.

• Cheri by Jo Ann Beard is published by Profile (£10). To support the Guardian and Observer order your copy at guardianbookshop.com. Delivery charges may apply

• Jo Ann Beard is interviewed by Catherine Taylor on 23 August in an exclusive UK event with Fane Productions Online