He is one of the world’s most famous novelists, with more than 150m copies of his fantasy masterpieces sold across the globe, but JRR Tolkien always dreamed of finding recognition as a poet.

Tolkien struggled to publish his poetry collections during his career, although he included nearly 100 poems in The Hobbit and The Lord of the Rings.

Now, half a century after his death, 70 previously unpublished poems are to be made available in a landmark publication. The Collected Poems of JRR Tolkien will be published by HarperCollins next month, featuring more than 195 of his poems.

His son and literary executor, Christopher Tolkien, had wanted his father’s poetic talent to be better known and, before his own death in 2020, worked on the project with two leading Tolkien experts, the husband-and-wife Christina Scull and Wayne Hammond.

Hammond told the Observer that there are “remarkably good” unpublished poems in the collection: “This will show even more Tolkien’s love of language, his love of words.”

Scull said: “The poems will add more to our view of Tolkien as a creative writer.”

They waded through a “great mass” of manuscripts and typescripts, some in Christopher’s possession and others in the Bodleian Library, Oxford, among other archives. The texts ranged from “beautiful calligraphy to the worst scrawl”, Hammond said.

During the first world war, Tolkien had been a signals officer with the Lancashire Fusiliers when he was posted to France and saw action on the Somme. In late 1916, he was invalided home with trench fever, a bacterial disease that almost certainly saved his life as his battalion was all but annihilated.

The unpublished material includes war poems, metaphorical works that are concerned with life, loss, faith and friendship rather than trenches and battles. Scull was particularly moved by an unfinished poem, The Empty Chapel, about a lone soldier hearing marching feet and drumming. “I found it very affecting,” she said.

In its extensive fragmentary drafts, Tolkien wrote: “I knelt in a silent empty chapel/ And a great wood lay around/ And a forest filled with a tramping noise/ And a mighty drumming sound/…

“O ye warriors of England that are marching dark/

“Can ye see no light before you but the courage in your heart.”

Tolkien’s humour emerges in a poem titled Monday Morning, where everything goes wrong for him, from slipping on soap to falling down stairs. It begins: “On Monday morning all agree/ that most annoying things can be./ Now I will tell you in this song/ of one when everything went wrong./ The sun was early shining bright,/ but not, of course, for my delight:/ it woke the birds who woke mama,/ who woke the boys, who woke papa;/ it came and hit me in the eye,/ though still I wished in bed to lie …”

Scull and Hammond struggled to make sense of a poem, titled Bealuwérig, that Tolkien had written in Old English. It features the name Bealuwearge, Old English for “malicious outlaw”, which recalls Tolkien’s fell creature in The Lord of the Rings, the Balrog, and the wolf-like beasts in The Hobbit called Wargs.

They were looking up words in Old English dictionaries, but could not find them – eventually discovering that Tolkien had been translating Lewis Carroll’s famous nonsense poem Jabberwocky into Old English, making up words to represent Carroll’s made-up words.

Hammond said: “Well, no wonder I couldn’t find the words in dictionaries.”

Each poem has an entry showing its development through various drafts, sometimes over decades.

In the introduction, the editors write: “Because his most commercially successful writings, The Hobbit and The Lord of the Rings, have had so many readers, and because they include between them nearly one 100 poems (depending on how one counts), Tolkien’s skill as a poet ought to be already well known …

“Many who enjoy his stories of Middle-earth pass over their poems very quickly or avoid them altogether, either in haste to get on with the prose narrative or because they dislike poetry in general, or think they do. It is their loss, for they are missing elements integral to the stories which help to drive their plots and contribute to character and mood.”