When the great library at Alexandria went up in flames, it is said that the books took six months to burn. We can’t know if this is true. Exactly how the library met its end, and whether it even existed, have been subjects of speculation for more than 2,000 years. For two millennia, we’ve been haunted by the idea that what has been passed down to us might not be representative of the vast corpus of literature and knowledge that humans have created. It’s a fear that has only been confirmed by new methods for estimating the extent of the losses.

The latest attempt was led by scholars Mike Kestemont and Folgert Karsdorp. The Ptolemies who created the library at Alexandria had a suitably pharaonic vision: to bring every book that had ever been written under one roof. Kestemont and Karsdorp had a more modest goal – to estimate the survival rate of manuscripts created in different parts of Europe during the middle ages.

Using a statistical method borrowed from ecology, called “unseen species” modelling, they extrapolated from what has survived to gauge how much hasn’t – working backwards from the distribution of manuscripts we have today in order to estimate what must have existed in the past.

The numbers they published in Science magazine earlier this year don’t make for happy reading, but they corroborate figures arrived at by other methods. The researchers concluded that a humbling 90% of medieval manuscripts preserving chivalric and heroic narratives – those relating to King Arthur, for example, or Sigurd (also known as Siegfried) – have gone. Of the stories themselves, about a third have been lost completely, meaning that no manuscript preserving them remains.

The study also addressed the question of how representative the surviving stories and manuscripts are. Medieval Irish and Icelandic narrative fiction seems to have survived far better than the English equivalents. One reason might be that the practice of copying texts by hand persisted for much longer in Iceland and Ireland than in England, meaning that any given medieval tale is preserved in more manuscript copies – and so protected, to some extent, against the inevitable loss.

The causes of loss were manifold, from fires and other disasters, to the decay or recycling of material on which texts were written, to censorship, incompetence and corruption. Throughout history, the most destructive of these forces was probably fire – and not just in the western world.

Michael Friedrich, a sinologist at the University of Hamburg in Germany, notes that the imperial library of China’s Han dynasty was largely destroyed by fire in the first century AD, during a period of internal strife. When a later dynasty tried to send another imperial library by canal to its new capital, most of the ships sank.

The largest repositories of books have always tended to form in centres of power, sometimes lending that power legitimacy, which makes them obvious targets during political upheaval, or just collateral damage when regimes change. As Italian literary historian Luciano Canfora wrote in the 1980s, in The Vanished Library, the upshot is that “what has come down to us is derived not from the great centres but from ‘marginal’ locations, such as convents, and from scattered private copies”.

There’s yet another problem: the sheer volume of texts. When it comes to Indian and Buddhist traditions, for example, the number of ancient manuscripts that have survived but are yet to be studied has been estimated at around 10m, though Friedrich says he has seen estimates as high as 30m. There simply aren’t enough scholars with the right expertise, including the necessary language skills, to do the work.

It’s tempting to think that after the advent of movable-type printing, which happened in Europe in the 15th century (and centuries earlier in China), literary erosion might have slowed down, simply because churning out copies became easier. But David McInnis at the University of Melbourne says that’s not necessarily true. For one thing, accidents continued to happen, as when rioters vandalised London’s Cockpit theatre in 1617, starting a fire in which all the theatre’s playbooks were burned.

For another, not everything that made it to the stage made it to the page. When plays were printed, it tended to be done cheaply with a single print run of around 500 copies, and these copies were often read to pieces – literally. As a result, McInnis says, we are probably missing the first edition of Shakespeare’s play Love’s Labour’s Lost, since the earliest known edition is described as a revision. All that’s left of another play we know the great man wrote, Love’s Labour’s Won, is its title.

McInnis estimates that the 543 plays that survive from 1576, when the first public theatres opened in London, to 1642, when the Puritans closed them, represent a fraction of all those produced. Another 744 that certainly existed have been lost, and hundreds more were probably written to fill the repertory calendar, of which no trace remains. Some plays were translated into German and performed on the continent by travelling English players, including works by Shakespeare and Christopher Marlowe. At least one play written for the English theatre, whose author is unknown, only survives in German – The Comedy of Queen Esther and Haughty Haman – and there may be others.

Unfortunately, we can’t console ourselves that the plays that do survive were necessarily the best, or at least the most popular. McInnis crunched the numbers based on the meticulous book-keeping of one London impresario in the 1590s, Philip Henslowe, and drew the following conclusion: “Lost plays performed at least as well as, and usually better than, the plays that have survived. They are definitively not inferior, they were good money-makers, and they have been lost for a variety of reasons that aren’t attributable to quality.”

In fact, literary historians tend to avoid the issue of quality altogether. The problem is that our criteria for judging literary talent have been shaped by the texts that have come down to us. Daniel Sawyer of Oxford University says there were certainly first-rate medieval writers in English whose works haven’t survived, but questions whether we would be equipped to judge those works, should any surface now. For English-speakers, Sawyer says, one writer casts a gigantic shadow over the rest: Shakespeare. Not only has he left his fingerprints all over our language, but he is the reference to which all other writers are compared.

Yet in his lifetime Shakespeare was responding to a rich and varied literary ecosystem. His contemporaries also recognised other greats – including a poet named Thomas Watson whose lauded plays have almost all been lost (only one survives, his version of Sophocles’ Antigone written in Latin). Who knows how we would judge Shakespeare – whom one contemporary described as “Watson’s heir” – had the full spectrum of English literature from his time, or the eras before and after him, survived. Who might the giants of world literature be, if we knew just what those 30m Indian manuscripts contained, or if the millions of others that were burned or mouldered away had survived. Greatness may sometimes be less a property of great minds than an accident of history.

Further reading

Shakespeare and Lost Plays: Reimagining Drama in Early Modern England by David McInnis (Cambridge, £29.99)

The Swerve: How the Renaissance Began by Stephen Greenblatt (Vintage, £12.99)

The Woman Who Discovered Printing by TH Barrett (Yale, £17)