The year 1594 was a fruity one in English literature, in part thanks to the influence from beyond the grave of Christopher Marlowe. Two of the dramatist’s most queerly themed plays made it into print that year: Edward II, about the medieval king’s passionate desire for his favourite Piers Gaveston; and Dido, Queen of Carthage, a dramatisation of the Aeneid that opened with a remarkable scene of erotic foreplay between Jupiter and his sexual plaything, Ganymede.

Yet even Marlowe’s vivid homoeroticism paled in comparison with the work of a very young and bold new poet who arrived on the scene that autumn. Richard Barnfield was just 20, recently down from Oxford, and ready to put to use an adolescence spent reading as much queer classical and vernacular literature as he could get his hands on.

In the title poem of his collection The Affectionate Shepherd (1594), Barnfield produced an eye-poppingly raunchy refit of Virgil’s second eclogue. Daphnis yearns for the exquisite Ganymede, not Jupiter’s pageboy this time but a golden-haired godlike youth who is well out of his league: “If it be sinne to love a lovely lad,” gasps Daphnis, “Oh then sinne I, for whom my soul is sad.” Daphnis relates acts of voyeuristic intrusion and daringly imagined fantasies of fellatio:

O would to God, so I might have my fee,
My lips were honey, and thy mouth a bee
Then should’st thou sucke my sweete and my faire flower,
That now is ripe and full of honey-berries

Barnfield reunited Daphnis and Ganymede the following year in a sequence of love poems he called Certain Sonnets, published as part of his next collection, Cynthia (1595). He kept the yearn and also upped the raunch: Daphnis ogles Ganymede as he swims in the Thames (where water god Neptune tries to cop a feel), and dreams of kissing the boy’s “coral lips” – a reverie that causes him to “speed” (orgasm) in his sleep and long to do the same “in deed”. In the middle sonnet of the sequence Daphnis outs himself as Ganymede’s lover, characterising the youth as “the perfect form of my felicity” and the cause of his heart’s romantic suffering. It was the most unabashedly homoerotic moment that English verse had ever produced.

And it was too much. There’s a reason you probably haven’t heard of Richard Barnfield. His queer poetry (which included an intensely misogynistic retelling of the Orpheus myth) sparked a controversy that may have cost him his station in life. There’s evidence that he was disinherited by his family, and after his astonishingly productive early years he disappeared from history. None of his work was reprinted under his name between 1605 and the 19th century.

But someone in the 1590s was paying attention: William Shakespeare. Judging by opinions that Shakespeare gave his dramatic characters, he didn’t think much of sonnets as a young writer. In Love’s Labour’s Lost, the worldly Berowne has no time for sycophantic love poems that make “flesh a deity” or a “green goose a goddess”. There was certainly something formulaic about the conventions followed by most sonneteers: all those glacial unreachable mistresses with sun-like eyes and alabaster skin, versions of Petrarch’s Laura sweeping along under English skies.

The sequence that was eventually published in 1609 as Shakespeare’s Sonnets did away with those tropes. In 154 sonnets, mostly written over a number of years after 1595, Shakespeare sensationally split apart the target of the poems’ “composed rhymes” (in the words of The Two Gentlemen of Verona’s Proteus). A mistress is the subject of about a fifth of his sonnets, but she isn’t a remote ice queen. She has sex with the narrator, a lot, and significantly for a poetic form in which the beloved’s rose-and-damask complexion was a conventional part of her allure, the woman has an appearance that suggests she is a person of colour (with tightly curled dark hair and skin that appears “dun”, or brown, compared to the snowy bosoms of Petrarchan heroines).

There’s still a gilded and distant lover in Shakespeare’s mix, however: a beautiful young man. The boilerplate terms of Elizabethan golden verse cluster around him – pearls, roses, fire and stars twinkle out in more than 120 poems that narrate, in impressionistic and overlapping ways, a giddily intense relationship between the two men. It shifts from the provision of avuncular life advice to obsessional desire to masochistic self-abasement (“Being your slave, what should I do but tend / Upon the hours and times of your desire?”). Shakespeare had realised how to make the sonnet much more than a vehicle for tacky idolatry: he queered the hell out of it.

And the person behind his innovation was Barnfield. It’s clear Shakespeare knew the younger man’s poetry: lines of influence linking Certain Sonnets and The Affectionate Shepherd to Shakespeare’s Sonnets have been traced in detail by modern literary critics. More importantly, no English poet before Barnfield had thought to invert the Petrarchan model and make the sonnet subject an exquisite fair youth.

We haven’t been encouraged to notice the connection. Partly that’s because we’ve been obsessed with discovering a real “lovely boy” behind the fiction, an Earl of Southampton or Pembroke flying undercover in the volume’s obscure dedication to a “Master WH”.

Mainly scholars have bent over backwards to suggest that same-sex sonnets were a standard genre in the repertoire of a late Elizabethan poet, and therefore devoid of real sexual feeling. As early as the 18th century, historians were peddling the story that “whole sets” of Elizabethan sonnets celebrated virtuous friendship between men – of which Shakespeare’s were simply an example. The celebrated editor Sidney Lee fibbed through his teeth when he told readers in 1905 that “hundreds of sonneteers had celebrated, in the language of love, the charms of young men” – without, of course, any erotic subtext.

They hadn’t, and those hundreds of sonneteers didn’t exist. Barnfield and Shakespeare were the only two Englishmen to publish same-sex love sonnets until the 20th century. Lee’s “hundreds of sonneteers” were decoys, a spray of made-up literary chaff designed to stop critical radar zeroing in on Shakespeare and Barnfield as exceptional artists linked by a shared artistic project: the queered Petrarchan sonnet, in which the untouchable beloved was regendered as a beautiful boy. Generations of critical discomfort with Shakespeare’s homoerotic sensibility conjured up a cloud of false witnesses, in the process seriously misrepresenting early modern queer culture and denying Barnfield his due as a major influence on Shakespeare.

In my book Straight Acting: The Many Queer Lives of William Shakespeare I put that right, resituating Barnfield as a significant early queer voice in the rich and complex cultural world of the 1590s. Like Shakespeare – like Marlowe – Barnfield was a pathbreaker, forging a new way to articulate queer feelings in a society that was still reaching about itself for the words and concepts that gave shape to sexual identity.

• Straight Acting: The Many Queer Lives of William Shakespeare is published by Sceptre. To support the Guardian and Observer, order your copy at guardianbookshop.com. Delivery charges may apply.