Michael Pollan first became interested in new research into psychedelic drugs in 2010, when a front-page story in the New York Times declared, “Hallucinogens Have Doctors Tuning in Again”. The story revealed how in a large-scale trial researchers had been giving terminally ill cancer patients large doses of psilocybin – the active ingredient in magic mushrooms – to help them deal with their “existential distress” as they approached death. The initial findings were markedly positive. Pollan, author of award-winning and bestselling books about botany, food politics and the way we eat, was born in 1955, a little too late for the Summer of Love. That New York Times story, however, was the beginning of an “adventure” that saw him not only explore the new research, but also detail the history of psychedelic drugs, the “moral panic” backlash against them and – partly through personal experiments with LSD, magic mushrooms and ayuhuasca, the “spiritual medicine” of Amazonian Indians – to examine whether they have a significant part to play in contemporary culture. The result of that inquiry is a compulsive book, How to Change Your Mind: Exploring the New Science of Psychedelics. This interview took place by phone last week. Pollan was speaking from his home in northern California.

Do you see this book on psychedelics as a departure in your writing, or part of a continuum?
Both, really. I have this abiding interest in how we interact with other plant and animal species and how they get ahead in nature by gratifying our desires. And one of those desires I have always been keenly interested in is the desire to change consciousness.

You propose the idea at one point that neurochemistry is perhaps the language by which plants communicate with us. Isn’t it more that magic mushrooms have evolved a clever way of making themselves invaluable?
They have. There is no intention involved, obviously. But evolution does not depend on intention. One strategy that these fungi seem to have hit on is manufacturing a chemical that can unlock these effects in the animal brain. Obviously some drug plants benefit us by relieving pain or boredom, but others do interesting things with consciousness.

Why do you think interest and research in psychedelics, LSD in particular, became almost taboo after the 1960s? Was it the cautionary tale of Timothy Leary [the lapsed Harvard psychologist, who evangelised for LSD and who Richard Nixon called “the most dangerous man in America”]?
When I asked scientists what led to the backlash, they would say, “Well, it’s too easy just to blame Timothy Leary”. But then they would go on to do just that. Would you have had the moral panic that engulfed LSD without Timothy Leary? I’m guessing you probably would. These drugs were finding their way into the counter-culture along other roads, the most obvious being Ken Kesey and his “acid tests”. But on the research side, it really was Leary who provoked the psychiatric establishment to turn against LSD. One of the things I am trying to do in this book, though, is to shrink the role of the psychedelic 60s because I think that overwhelmed the whole discussion. The history of this experimentation goes back thousands of years to very ancient meso-American societies, to the ancient Greeks.

In those cultures, it seems clear these substances were treated with enormous respect – religious rites grew up around them, and they were in the hands of the elders. I guess that is one of the arguments for your experimenting now, in your early 60s?
It’s true, these things were always surrounded by ritual. If you want to point to one mistake of Leary and Kesey it was giving up on the guide, aka the elder. Their eagerness to democratise the benefits of LSD as they saw them led them to say, “Take this in any circumstance. You want to be out in a park with 25,000 people doing this? OK.”

I talked to Kesey once. His argument was that they believed they could see a revolution in consciousness happening before their eyes…
That was the argument – and arguably it did work. Part of the culture has turned against the 60s, but in reality most political arguments are still about the 60s. Trump is about that. There is a tremendous reactionary force in our politics that you can trace back to things that happened then – I’m talking about feminism, civil rights and all the changes in social relations that were pioneered. We are still trying to digest that great bolus of change.

It seems to me that we remain in a paranoid moment rather than an experimental or a playful one…

I think these things swing, but the pendulum is a little changed on every swing. I have been talking to some audiences of people in their 20s and 30s, and there is a lot of hopefulness and idealism. Things survive in interesting ways. Some of these people are very interested in psychedelics but they tend to be very mindful of the importance of the guide or the circle or the group, and not be careless about it.

How easy was it to turn off your inbuilt scepticism when you found yourself, as you describe in the book, off-grid in a yurt, listening to pipe music, and being given LSD by the son of a German Nazi?
Yeah [laughs]. It was not easy for me to go to the places that this took me. I am not a psychonaut, or I was a very reluctant psychonaut. Every night before I was going to have one of these big experiences was a sleepless night of mental ping-pong. Part of me was saying, “Are you crazy? You are going to go to a place where they cannot even call 911, with a person you barely know. You are going to put your life in his hands. You could actually go crazy.” And then the other side was, “Aren’t you curious? Don’t you want to try this?” And also “You have a book to write”. It was agonising.

It seems like it was also out of character.
Absolutely. By accident of birth and temperament, I am more a child of the moral panic about psychedelics than a child of the 60s. I had bought the propaganda that these were dangerous substances. That was the baggage I was carrying into this experience. In that situation, your ego has command of your rational faculties, so it summons really good arguments.

It suggested that you go to visit your cardiologist first, for one thing…
Yes. Though to my surprise he gave me the green light for everything but MDMA [the drug ecstasy]. But then, as soon as I had passed the point of no return as it were, and ingested the mushroom or put the LSD blotter paper on my tongue or drained a cup of ayuhuasca, I let go of ego more easily than I thought I could. And even the new-age hokeyness of the ceremonies – and I am allergic to woo-woo which I am surrounded by here in northern California – I just accepted the terms of the world I had entered. I think that was a testament to the guides I was working with.

Was your wife [artist Judith Belzer] supportive of what you were doing?
Judith was very supportive. For one thing, she liked the change of direction in my writing – she may have been sick of hearing me talk about food. In terms of the experiences, I think she had some trepidation about what effect it might have on what had been a long and happy marriage. She was worried I would be changed by this – but as the experiences unfolded I think she was reassured that I was perhaps being changed for the better. It is not for me to talk for her. But I would come home and have dinner and explain what had happened and she was really helpful in making sense of it, joining the dots.

I have never used psychedelics, but when I have spoken to people over the years who have, I’ve been struck by how, although they almost universally hold the experience in a special place, they have often not carried on experimenting. Is that how you feel?
I share your impression of that. Just going out and talking about this now prompts many people to tell stories perhaps for the first time in 30 years. One of the things I hope this book will provoke is more people coming out and talking about how their experience with these drugs changed them. Did you see the Guardian profile of the physicist Carlo Rovelli? Wasn’t it fascinating how he suggested that taking LSD opened up his mind to some of the concepts of modern physics? How it helped him understand that the world as it presents itself to our senses may not be the only one. That is a fundamental psychedelic insight? What is interesting to me is how that insight – that there is an unseen world – can lead you both in a religious direction and a scientific direction. These drugs don’t interpret themselves. They open up material to be interpreted.

We stand on the edge of environmental catastrophe and quite possibly geopolitical catastrophe. Do you have any sense that the demand and desire for shifts in consciousness has a direct relation to those anxieties?
That’s an intriguing theory – that when human society is approaching a precipice and there is a need for new thinking, it can arrive through these experiences. I don’t know how you would prove that. But I do know one of the things that many people emerge from psychedelic experiences thinking is that if we all had these experiences there is no way we could treat nature, or each other, as we do. I don’t know that this would lead to benign social change. I put one name out there: Charles Manson. But it is curious to me that what I see as the two greatest threats – environmental crisis, and [political] tribalism – these drugs directly address both those mindsets. They undermine our tendency to objectify nature, to think of ourselves as separate from it. They undermine tribalism in that people tend to emerge from these experiences thinking that we are all more alike, all more connected. How do you make use of that? We don’t have models for prescribing medicine to a culture – although we do it with alcohol I suppose.

You seem persuaded in the book that psychedelics certainly do have a role for terminally ill patients and in treating depression and addiction.
There is enormous potential for these medicines to address mental illness of various kinds. Based on the research we have and the people I’ve talked to, it would be foolish not to explore this as carefully as possible. A lot of this adventure started with my interviews with cancer patients, and here were people confronting their mortality, at a time when our culture offers very little, particularly to people who are not religious. Many of these people were stone-cold atheists, and [the drug experience] really helped them reconceive what it means to die; it made death seem less catastrophic and in many cases eliminated their fear. Now whatever interpretation you put on it, that is extraordinary. It seems to me if we have a tool that helps people die with less terror and less anxiety that’s something we need to explore. And in terms of addiction there is lots of preliminary evidence that says this can be useful, we need to see how it holds up in phase three trials.

You must have heard plenty of naysayers too.
Yes, but not as many as I thought. When I published the New Yorker story [about the trials that gave hallucinogens to cancer patients] that led to the book, my editors got cold feet in the weeks before it came out, because it was a pretty positive take on a drug that at that point hadn’t even gone past stage two trials. They asked me to go and find more naysayers. One of the first people I called was Tom Insel, who at the time was director of the National Institute of Mental Health, the leading voice of the psychiatric establishment in America. I was expecting him to say something devastating that I would put in the first page of the article and my editors would be happy. But he said instead, no, this research is really interesting and important work. I thought: “Oh, shit! I’ve got to keep looking.” I am actually surprised there hasn’t been more pushback from scientists, but I haven’t found it.

And we also haven’t heard from the pharmaceutical industry. Are they looking at this stuff?
Presumably. They look at everything. But on the other hand, they have a preference for molecules you can patent and for pills you take every day of your life, rather than once or twice. That is another question – how do you profit from psychedelic-assisted treatments?

Since you started this experiment, do you feel there has been a fundamental shift in your own self?
Fundamental would be too strong, but a definite shift. It is hard to articulate. But I think I have a different relation to my ego. I have a sense of this character who inhabits me, and is very important part of me – he got this book written for example – but that he is also neurotic and defensive and gets in the way of certain experiences. And I think I am better able to put him in his place. To take one example, my dad died in January and I was with him for the last 10 days or so. My wife felt that how I was able to approach that was different to how I would have done a couple of years ago. She felt I was much more open and available to him and to my mother.

Do you carry that feeling into your own sense of mortality?
I don’t think I have gotten there yet, but it has given me some interesting ways to think about it. How do we define this self that is going to be extinguished? Is it just a bag of skin and what is in it? Or can we define ourselves in a broader way to take in our loved ones, our offspring or our community even? To the extent that you can do that death becomes less cataclysmic. You asked earlier if I am going to continue experimenting. The answer is I don’t know. But I do know people who seek out one of these experiences, always with a guide, on their birthday each year. I can imagine that.

When’s your birthday?
6 February. Call me then and I’ll let you know how it’s going.

How to Change Your Mind (Allen Lane, £20) is published on 18 May. To order a copy for £17 go to guardianbookshop.com or call 0330 333 6846. Free UK p&p over £10, online orders only. Phone orders min p&p of £1.99