In June last year, three days after the fire at Grenfell Tower, John Boughton wrote a restrained yet furious post on his blog, Municipal Dreams, about the legacy of refusing to treat council housing with the dignity it warrants. “For almost four decades, we have been taught to see public spending as a bad thing; ruthless economising as a virtue … Public investment enriches lives; here it would have saved them. The best memorial to all those who have lost their lives in Grenfell is that we as a nation choose collectively to invest in safe and secure public housing for all who need it.”

This, Boughton’s first book, isn’t a reworking of the blog but an original work whose purpose is to argue passionately for that idea. For the past few years his writing has been an elegant and compendious ongoing exploration of Britain’s social history through its council estates. The book celebrates an era during which dreams of shelter and security for all – not just those who could afford to purchase it – were in large part made a reality, and asks us if we oughtn’t to consider reviving that dream before it gets destroyed completely.

Some five million homes were built by and on behalf of local authorities in the first three-quarters of the last century, reaching a point where, at the end of the 1970s, around a third of all households, more than 40% of the population, lived in council housing. The gargantuan political and physical effort this took, Boughton argues, transformed society largely for the better. It saved an untold number of lives and gave dignity and security to millions.

Britain’s first municipal housing, built at Vauxhall in north Liverpool in 1869, was established on the advice of Liverpool’s first medical officer William Duncan, who was among the first to establish a link between insanitary housing conditions and poor health. One third of the working-age population – 86,000 – – 86,000 people – were squashed into just over 2,000 cramped blocks, with a further 38,000 living in cellars. In Ancoats, central Manchester, a report by the physician John Thresh concluded that 3,000 to 4,000 people were “dying annually from remediable causes” – causes remedied, in other words, by decent housing.

Piecemeal attempts at finding ways to “rescue” better-off – and socially better-regarded – working-class tenants from slum conditions came in the Victorian period from industrialists and investors who benefited from “5% philanthropy”, named after the promised rate of return on house-building schemes for artisans and clerks who tended to earn enough money to pay the higher rents.

It took two world wars for local and central government to address slum clearance and to commission mass housing schemes which met the needs of growing cities. Quality and size went up and down according to whether Labour or the Tories were in government: Labour’s first years in power from 1924 were marked by improved minimum standards for council dwellings, which were reversed by the post-crash National government in the 1930s, leading to many pinched, uniform interwar flats.

Nye Bevan famously instituted luxurious minimum standards for his postwar council houses, which averaged 1,055sq ft for a three-bedroomed house with a bathroom and separate lavatory. He condemned the interwar estates as “castrated communities” –the housing meant solely for working-class people who were taught they should be grateful for what they got – rather than council housing so good that anyone on any income would want to live in it. But Harold Macmillan, who became housing minister in 1951, built a larger number of smaller, cynically named “people’s houses”.

One word Boughton keeps coming back to in his account of “the rise and fall of council housing” is “decency”. It can be used romantically, he acknowledges, as a kind way of overlooking how brutal living conditions can come to brutalise those who endure them. But it also sums up his view of what, and whom, council housing is for: a sense of societal decency and fairness, in which every member enjoys comfort and safety.

Tories have always tended to see it as compensatory and temporary: you use it when you can’t buy your own, and only until you can buy your own. Historically, Labour tried to maintain a sense that council housing was for anyone who wanted it, a tenure as valid as ownership. We know which view has won out since the “right to buy” became law in 1980. The massive popularity of discounted home ownership among sitting tenants seemed to confirm the emerging neoliberal view that all anyone wants is individual control over individual goods.

Yet the massive shift to private renting in the past decade – at best, unsatisfactory, at worst, dangerous – partly resulting from a diminished stock of council housing has brought more people into contact with the sharp end of housing insecurity and unaffordability. Housing has been rising steadily up the political agenda since the 2008 crash; the causes and consequences of the Grenfell fire made that concern a matter of urgency. There couldn’t be a better time for this book.

• Municipal Dreams: The Rise and Fall of Council Housing is published by Verso. To order a copy for £16.14 (RRP £18.99) go to guardianbookshop.com or call 0330 333 6846. Free UK p&p over £10, online orders only. Phone orders min p&p of £1.99.