Last Monday, five of the world's foremost science writers gathered in London to learn if they would receive the Royal Society's Winton prize for science books 2012. Prior to the evening's awards, we gathered them in the society's library to talk about reporting on what one of them, James Gleick, called the "very edge of what I'm able to understand". Writers such as Gleick play a vital role in how we comprehend the world around us. And as science discovers more and more about our universe, its theories and findings become more and more technical and data-driven.

So the role of scientists and science writers, such as the five assembled here, in turning this complex work into accessible, illuminating prose becomes trickier and more vital. The sixth nominee for the prize, Nathan Wolfe, author of The Viral Storm, was unable to attend on account of being in the Congolese jungle. No live satellite link-up was needed, however, since the judges awarded the prize to Gleick for his book The Information.

Ian Tucker: Why is popular science reporting important?
Joshua Foer: When the Royal Society was founded in 1660, it was still possible for an educated person, a polymath, actually to know something about everything. Today, that is not possible. Steven Pinker might be a great cognitive scientist but I bet he can't explain how they discovered the Higgs boson.
Brian Greene: He just explained it to me earlier and he did quite a good job.
JF: That speaks to why we need great interpreters more than ever. And what we do becomes more and more important because as science becomes more esoteric it requires people to help the rest of us to understand it.

When you are writing where do you set the difficulty dial? Do you want your readers to finish your book in one sitting or work hard at every sentence to glean some insight from it?
Steven Pinker: Before I wrote my first cognitive book, I got a bit of advice from an editor, which was probably the best advice I ever received. She said that the problem many scientists and academics have when they write for a broad audience is that they condescend; they assume that their target audience isn't too bright, consists of truck drivers, chicken pluckers and grannies knitting dollies, and so they write in motherese, they talk down. She said: "You should assume your readers are as smart as you are, as curious as you are, but they don't know what you know and you're there to tell them what they don't know." I'm willing to make a reader do some work as long as I do the work of giving them all the material they need to make sense of an idea.
BG: The ideal thing is that readers can take in a book at a variety of levels. If they really want to dig in deep to understand every last thing, including the end notes, then God bless you, that's fantastic! But others who just want get the gist of it, to let the ideas wash over them, I hope the text has enough momentum to propel them through that. It's a hard balance to strike, but I think people appreciate it when you are at least close to that target.
Lone Frank: A lot of people will say don't use difficult technical terms, but basically you can explain anything to people if you do it by good writing using plain words.

Has anyone omitted anything because they found it too difficult to explain?
BG: Ever? Always. I agree in principle that you can explain everything, but you try to explain [symmetries] in algebraic geometry to a lay audience. Good luck. There are things that really are too hard to understand without a technical training, but the art is still leaving enough of the heart of the subject so you haven't eviscerated it and you've done justice to the subject.
JF: Ultimately, everyone in this room is on some level an entertainer. We are competing for readers' attention against blogs, video games and movies. What I'm trying to do is tell stories that can take people from place A to place B, not just in a narrative arc but in terms of their understanding of a subject. It can be tremendously rewarding to be taken on a journey like that.
James Gleick: I don't know whether this is confessing or bragging, but I'm often at the very edge of what I'm able to understand myself. I'm writing books about things that I care about, telling the stories that I think matter to our culture and our culture is more and more about scientific things. My book The Information is in some ways not a science book at all, but in writing it I had to grapple with some things that were quite technical. So for me to get to grips with it often requires asking smart people a lot of dumb questions.

The lay person might not realise there are particular issues with science writing and lump all non-fiction together.
JG: I'm with the lay person. I don't think I'm typical here; everyone but Josh [Joshua Foer] is or was a scientist. I backed into it by way of journalism; I never meant to be a science writer. I resist the idea.
LF: I agree. I think a lot of the time what distinguishes science writing from non-fiction is that is quite boring, quite formulaic.
SP: As the mindset of science is applied to other fields, the distinction is going to harder and harder to make. My book [The Better Angels of Our Nature] is an example. It's partly a book of history, partly a book of cognitive and neuroscience and it has also been nominated for a history prize. I look at historical trends and in my view if you talk about a trend and you use the words decrease or increase there should be numbers attached. So I report history that can be quantified and summarised in graphs. Not all historians are on board with that approach to doing history. To the extent that they are, the distinction between science and general non-fiction is going to erode. Also if one wants to explain historical phenomena as I do – that is not one damn thing after another, but why things seem to go in a particular direction – if you have a phenomenon and you are trying to explain it in terms that are more general and simple, in a sense you are doing science.

Learning is associated with difficulty in most people's minds. Do you think sometimes it has to be hard, otherwise it's not considered science?
SP: I'll give you an anecdote that might give you the difference between the mindset of the scientist and the humanities scholar. I once went to an interdisciplinary conference with scientists and humanities professors. At the end of a talk exploring a painting, the speaker said: "Well, I hope to have complicated the subject matter in several ways." I thought, that's the difference between a scientist and a critic – the scientist would say: "I hope to have simplified the matter in several ways."
LF: A lot of readers have an expectation that everything should be easy these days; you shouldn't work for absolutely anything, so you won't read a book if it's a little too difficult; you just throw it away and find something else.
JG: That's a depressing thought. I hope it's not true. I certainly don't write as though it's true. I think people like to be challenged. And it doesn't have to be about science – there are books on history and of literary biography that are challenging. If you're reading a book of any length it's because you want to.
LF: I think you're talking about the 1% and not the 99%.
JG: You know, in the book business it is about the 1%.
BG: That's a lot of sales; in the US that's three million.
LF: I think there's probably a cultural difference. Where I come from, Denmark, science is just not valued at all, we hardly have any science writing. For example, Richard Dawkins cannot get his latest book translated because there's no market for it. That tells you something.

There seems to be a mini boom going on in science books at the moment. Would anyone like to speculate why?
BG: Thankfully, my experience has been so different from what you described in Denmark, which is frightening. Obviously, it's a very self-selected group that I encounter but I think there is a growing body of people that really wants to know what's happening in the world of science.
SP: Also, more and more, we educated people live in a world that is described by science. People don't believe that the world was created 5,000 years ago, at least the kind of people that we try to attract to buy our books. Educated people will accept that we are evolved from primates, that our mental life depends on the functioning of our brains, that we're subject to illusions, fallacies and biases. These are deep existential questions and it's science that is posing and answering them.
JG: You've just defined an entire American political party as uneducated!
SP: Yes, indeed.
JG: It used to strike me as obvious, that the question didn't need to be asked: of course we care about science, of course we understand that science is what explains the questions we most care about finding answers to. The scariest thing is the opposite tendency in our country (where four of us live), where suddenly it seems to me people are increasingly antagonistic to the notion that science is the place to go to answer these questions. I didn't see that coming. When I started writing about science it didn't occur to me that that would ever be controversial.
SP: It might be in that America one of the two political parties seems to defiantly oppose the world science view. But I suspect that isn't the best way of understanding it, because they still look for oil using the assumptions about the age of the Earth that we all believe in; when they get sick they go to a doctor and they worry about the evolution of drug resistance just as we do. They're not Amish, they don't return to the land. So in a sense they have already bought into the scientific world, but there are just a few highly symbolic issues that define your moral and political identity that they stake out a position on, and I think that is very different from scientific ignorance. In fact, one study done by a former graduate student at my department at Harvard showed that people who endorse the theory of evolution don't understand it any better than those that deny it. We shouldn't confuse the moralisation of a small number of hot-button issues with hostility with the scientific world view in general.

How do you judge a successful book? Sales? Feedback from academics?
BG: All that's part of it but one thing I ask myself is what it would have been like to write a book 20 years ago when there wasn't the ability of someone just to send you an email? When an eight-year-old kid writes to you with questions and is excited about the ideas, it's not anything to do with sales or overarching impact, but that one feels good when one person is so excited about your book.
JF: I don't think I would be sitting at the table if it wasn't for the three of you guys writing books that I read in high school that deeply impacted me – Chaos [by Gleick], The Language Instinct [Pinker] and The Elegant Universe [Greene]. All the books I read in high school shaped how I thought about the world and what I wanted to do with my life. I don't think you recognise the ways in which you affect people that you don't hear about until years later.

Joshua and Lone, you've both written books that thread science through your own experiences . Were you worried about treating your subject in a less objective way?
LF: In a book about personal genomics, it's very hard to go out in the third person and ask: "How did it feel to get this information?" You might as well go and get it yourself. My experience is one experience; somebody else will handle it differently. So I just use myself as the guinea pig to tell both sides of the story. It's also a way of drawing some people in. They will bite on to the personal story and get the science along with it.
JF: I'm only interested in writing insofar as it's a tremendous vehicle for exploring questions that I'm interested in and hopefully finding answers to those questions. I wasn't supposed to be in my book; that happened a little way down the line when my interest in the subject sucked me in to a degree where it would have been impossible for me to tell the story without being a part of it. So it's a happy accident that I'm one of the main characters. I can't imagine writing a second book and not similarly being that deeply engaged.

Brian or Steven, would you write a book in the first person?
SP: I did a personal essay on having my genome sequenced several years ago. Also, I can imagine writing a book on the pitfalls of recounting life from the first person's point of view, drawing on what a psychologist knows about how self-serving and distorted our memories are.
BG: Chapters of The Elegant Universe are written in the first person. Initial drafts were not because the concern was it would feel too self-serving to write about your own work in a book for the general public. So the first draft had no names; it was just the ideas and it was utterly crap. So I went back and I put it all in, who had done what, the arguments about this, the decisions about that. Then, when it came to my own work, it was natural that, since I was involved in it, I told it from the first-person perspective. If you don't tell the story, it's not going to work.

Analogies. Is it better to make do with an accessible but maybe imperfect analogy rather than to describe in detail in possibly an impenetrable way?
SP: Analogy is enormously powerful. In fact, one could argue that we understand everything except for the physical world of falling objects by analogy. If you look at our language it's almost all metaphorical. But, there is a difference between literary metaphor and scientific analogy, and that is in a literary metaphor the more connections there are between the figure of speech and the thing in the world the richer and more wonderful it is, and in the scientific analogy if there are too many ways in which you can relate the analogy to the world, that makes it a bad analogy, not a good one. Analogies have to be chosen and explained carefully. You've got to point the reader to the correspondence, point for point between the thing in the world you're explaining in terms of your analogy. To be whipsawed between one analogy and other so you don't know what point is doing the work, that's what can make an analogy misleading.

I read somewhere that analogies are like ill-fitting coats – the most important parts are covered, but some may protrude and may restrict movement.
JG: I don't buy that! Anything can be done badly, but I also believe that analogy is the way humans learn and explore our world. It's true at some level that a physicist will say that the language of nature is mathematics, but I also believe that any physicist in creating his or her own understanding of the world is automatically thinking in terms of analogies. I believe that any scientific model or theory is a kind of analogy, which is to say imperfect, flawed by definition and at least incomplete. It's a model, it's not the world itself.

Has anyone here tried writing a novel?
SP: I'm married to a novelist so I recognise my incompetence.
BG: I published a short story that had real science in it. It was about time travel. It was one of the most fun things I've done. There's a certain kind of freedom that comes with the fictional aspect. The fact that it's tethered to reality by real science gives it a sense of stability that I would have trouble finding if I was just making it all up.

Are science writers reporting on the frontiers of knowledge or imagination?
LF: A lot of people don't realise that what comes out of science really affects the way they think. They think that culture comes out of philosophy, plays and theatre and stuff, and science just produces gadgets. I really want to show how the knowledge we have from science shapes our culture.
BG: I think it's imperative kids recognise that science involves as much creativity as any other so-called creative discipline.
JG: I don't think anybody at this table would be writing about science if we bought into the idea that the process of science was a matter of rote and grunt work. All five of us have focused on imagination and creativity, not just as the occasional accidental part of the scientific process but as the things that make it work, make it exciting.
SP: The only proviso to all of this is that in science it's not enough to be imaginative and creative, it also has to be right. There are plenty of very imaginative people history forgot, because their beautiful, elegant scheme didn't fit the facts.

How has the formal, technical way scientists write journal papers affected popular science writing?
BG: I was looking back over some quantum mechanics papers from the 1920s and in one article the scientist described an accident in his laboratory when a glass tube exploded, a nickel got tarnished and he heated it to get rid of the tarnish – he went through the whole story himself in the technical article. You don't really see that much these days. I don't know if that is a one-off example, I haven't done an exhaustive study, but have journal articles moved away from telling the story of discovery to just a more cut-and-dried approach?
SP: They have; I think that's been documented. There is scientifically a problem with that, as opposed to narrating what happened. The problem is that since you're under pressure from the journal editor to tell your story leading up to your conclusion without talking about all the blind alleys and accidents, it actually distorts the story itself because it inflates the probability that what you discovered is really significant. If you tried 15 things that didn't work and one thing that did work and didn't talk about the 15 that didn't work, then the statistic that makes it significant is actually mistaken. The statistic has to be computed over all of the experiments you ran, not just the one that happened to work. In the social sciences especially, we're seeing that there's a lot of damage done by the practice of only reporting the successes and telling the story as if it was a straight line to a successful result.

Are scientists their own worst enemies when it comes to communicating their work?
JF: People like me probably should be unnecessary in the whole process. Ideally, scientists would be great communicators and there wouldn't be any place for people who are non-experts to step in and put together a cohesive story about the work that scientists are doing.
JG: But those skills don't have to go together. You can be a great scientist and not have the slightest interest in communicating; Isaac Newton was a lousy communicator. I think we should just think that we're lucky that someone like Brian Greene is not only a great scientist but also a great communicator. I think that's more exceptional than typical.
JF: But I wonder whether scientists wouldn't be better scientists if they were better communicators?
BG: There are some who certainly think so. There's a new institute at Stony Brook that is geared toward having science communication as part of the training for graduate students of science. It seems reasonable to me that if people could communicate even at the level of scientist to scientist, more freely, more articulately, it might spark more things.
JF: What you're supposed to be doing in a science book or popular article is distilling, finding what is essential and communicating that. That's not just an act of storytelling, it's an act of thinking and it requires a kind of clarity of communication that not just the scientists but academics in general have moved away from and I think it makes them think less clearly.

Steven and Brian, do you feel that your academic colleagues take you less seriously because you write popular books?
BG: I was afraid of that early on, at least with my first book. But I found that at least to my face, the vast majority of people in the field of string theory felt that they were in this little corner of science that nobody knew much about and they just liked the idea of getting it out to the general public.
SP: My experience is the same, in that I don't want to let myself think those thoughts – because it would too easily furnish me with an excuse to blow off people who are criticising me and give me an excuse not to take the criticism as seriously as I ought to as a scientist.