Is the BBC in danger of meltdown in the accelerating climate of change in broadcasting? If so, how could that happen to an organisation still thought of as an immovable pillar in our culture and widely praised around the world as a big player?

Charlotte Higgins’s This New Noise: The Extraordinary Birth and Troubled Life of the BBC is an account of an organisation that embraces so many aspects of this country’s life, traditions and personality that it seems to represent the British character itself. The book could scarcely be better or better timed. It is elegantly written, closely argued, balanced, pulls no punches and yet wears its respect for the BBC on its sleeve.

It is particularly welcome as a corrective to the recent, in my view, distressingly inadequate volume of BBC history by Jean Seaton. In his review of Pinkoes and Traitors, TV executive David Elstein pointed out the inexcusable number of errors and omissions. There are promises it will be revised. There are those who think it would be better pulped. It is equally distressing to the BBC’s admirers that the current hierarchy in Broadcasting House has endorsed it. Surely they must think that the BBC deserves better than that?

Higgins has come to the rescue. She has employed a method long thought to be dead and gone. Through interviews and character sketches, she tells us a story of great men. The strategy is a perfect fit. There have been great women along the way, from Hilda Matheson, who worked alongside Reith (and coined the phrase “this new noise” to describe the unfamiliarity of the sound of the wireless), to Grace Wyndham Goldie, in the heyday of Lime Grove in the 1960s. And there were Catherine Dove, Caroline Thomson, Patricia Hodgson and many others. There is also a raft of women now on the staff of the BBC, some of whom will be serious contenders when Tony Hall retires. But so far it’s been the preserve of the great white male.

Higgins includes some chairmen and some programme-makers, but the core of her book is built around the occupiers of the office of director general, who is also editor-in-chief. This strategy yields riches of coherence, liveliness and a clear sense of truth to BBC history. Perhaps it has been its ceaseless growth, extension and reinvention that gave the corporation something of the Plasticine nature of an object constantly capable of being remodelled even when 25,000 strong. A director general could make the BBC in his image.

Lord Reith was its first inventor and shape-setter. When the BBC has strayed from Reith it has faltered. When it has the convictions of that tormented, Caledonian-giant engineer it has prospered. There is not a DG who has not felt judged by Reith. Reith was Genesis and all subsequent DGs were begat by him. Reith gave the BBC its grand purpose. “Its status and duties should correspond with those of public service… the BBC should be the citizen’s guide, philosopher and friend… and help to show that mankind is a unity and that the mighty heritage, material, moral and spiritual, if meant for the good of any, is meant for the good of all.”

This has been translated into more contemporary language over the years, but when the BBC loses sight of it, it will be finished as a unique institution. This and other declarations from Reith are in effect the BBC’s Magna Carta.

There is the central question of independence, well covered by Higgins. 1926 was the crunch, the General Strike, when the BBC was the only conveyor of news in the land (apart from the government propaganda sheet, the British Gazette, and the TUC’s the British Worker). Winston Churchill wanted to commandeer it and was repulsed. Stanley Baldwin, the prime minister, broadcast to the nation from Reith’s house and was coached by him. Ramsay Macdonald, the leader of the opposition, asked leave to broadcast his point of view. Reith supported him, but the government refused permission and Reith had to back down. He consoled his conscience by putting out statements from the TUC. A sort of impartiality was maintained but a sort of independence was fudged and that fudge has been the bane of the BBC ever since.

The second world war sealed the BBC into the nation. It also established the BBC around the world as a teller of truth and gave to this country an imperial prestige that has outlived colonial conquests.

Then the curtain lifted. Hugh Carleton Greene breezed in from journalism and, with panache, ushered the BBC into the exhilarating liberation and merry-go-round of the 60s. There was That Was the Week That Was, new plays and other programmes by the eager grammar school intake – Dennis Potter, Alan Bennett, Ken Loach, David Mercer and others. And there was Greene’s careful wangling of a second channel that proved to be vitally refreshing.

Greene licensed Grace Wyndham Goldie to run a talented nation state in the invigorating scruffiness of Lime Grove. It was to be from this that Panorama, Tonight and Monitor grew, as did the reputation of a future DG, Alasdair Milne, whose career brought into sharp focus the fight that had been going on since 1926. This brilliant man and born leader went into a BBC board lunch and came out before a morsel had been eaten to tell his PA that he had been summarily sacked. The executioner was Marmaduke Hussey, placed in the chair in 1987 by Margaret Thatcher to rid the BBC of its turbulent priest. The programme they picked on (there were several bold efforts to choose from) was a well-balanced film about Northern Ireland. But Margaret Thatcher was against any publicity given to the IRA and with a strike worthy of the mafia Milne was out. Whose BBC was it now?

John Birt (DG 1992-2000), who came from ITV and like Reith was an engineer and a serious public service broadcaster, says in this book that he found the BBC “unmanaged and ill-disciplined in a way I would not from the outside have thought possible”. The sacking of Milne had led to a severe wilting of purpose.

Birt, with characteristic intelligence and skill, reorganised it and used the reorganisation to fend off the government of the day. He was bombarded with criticism, most dramatically from Dennis Potter, and he got a cruel press, but as Christopher Bland, the BBC’s chairman of the day, says in this book, “strategically he was remarkable”. It emerged stronger.

Birt’s successor was another ITV man, Greg Dyke, who was the most popular DG in its history. His talent for nurturing talent in others built on Birt’s achievement and between the two of them they gave the BBC the ring of success and achievement.

Then the long arm of 1926, as it were, struck again. The Gilligan/Kelly affair brought Greg Dyke into direct conflict with Alastair Campbell and Tony Blair (whom Dyke had supported for many years). And once again the BBC board struck at a board meeting at which Dyke “just happened” to be without any allies and he was effectively sacked, although he chose to beat them to the punch by resigning.

Mark Thompson, a BBC man who followed him, strove to keep the image of the BBC he had inherited. His licence fee settlement, which seemed canny at the time, burdened the BBC with the World Service, which, together with the broadband rollout and the Welsh language commitment was more government policy than a BBC remit. But Thompson thought it was worth the trade-off. The storm clouds were approaching and Thompson battened down.

After the short interregnum of George Entwistle, Tony Hall was appointed in 2013. He had run BBC News under Birt and then gone on to be a successful director of the Royal Opera House. He has set out his stall and once again we can see the light coming from Reith. Hall’s strategy is dominated by a determination to ensure the BBC’s long-term survival.

Now the horns are locked. The BBC is fighting on several fronts at the same time. Military historians say that this is as bad as it gets.

Next year, the government’s key weapon, the BBC charter, will be on the table. Unfortunately, it is widely accepted by the current government that the BBC is too leftwing and this was reinforced by Thompson. The government is fielding John Whittingdale, a rightwing Conservative who knows a great deal about broadcasting. I think he has no intention of being the man who destroyed the BBC, but he will undoubtedly be tough on it. He doesn’t have to do much. He only has to freeze the licence fee again to cause considerable damage to the current BBC.

And the licence fee depends on the possession of a television set, and an increasing number of especially younger people do not watch television on a television set. Other individually serving systems proliferate. No TV set. No fee.

Meanwhile, the BBC’s internal troubles are serious. There is little doubt that when the Savile report comes out, BBC management will be heavily criticised and the claims of those who were abused could be vast and for years ahead a steady reminder of the BBC’s laxity.

There is also criticism of BBC management across the board. Hall promised to cut it right down but so far has only made a start. The complaints against the BBC are that the process is increasingly slow, complex and over-costly for independents. There is also still a grievance at the size of salaries and payoffs (which Hall has tackled) and a continuing sense of entitlement.

This criticism comes from people of the stature of John Lloyd, John Tusa and David Attenborough, who in Higgins’s book says: “Broadcasters should be the cream of thinkers in society who have been given by the BBC a platform on which they may speak, but the BBC doesn’t believe that now.”

However, in my current experience of working there, there is still a good core of people who do believe that. Perhaps I am privileged by working for Radio 4, but the people with whom I work are in every way the true heirs of those I met in radio in 1961. Upstairs, there is no doubting the quality and ambition of (to name only three) Hall, director of television Danny Cohen and Radio 4 controller Gwyneth Williams.

I must admit that the BBC could also address yet another battalion of critics that bemoans the fact that the BBC news, nationally and locally, makes up more than 50% of the news received by citizens of this country. People respect it. Those six pips seem to be the pulse of the nation. But there is a case to be made for more varied voices inside the BBC itself. Perhaps this could be achieved by separating radio and television.

The BBC relies increasingly on one of the great success stories of recent years – independent film-makers. These have now become essential to the BBC and vice versa. It is a positive and highly successful pact. Yet when the BBC puts out their programmes is it any longer the BBC of old? Is it not becoming a publisher? And if it is becoming a publisher, does it need more than 20,000 staff, dozens of buildings and all the other accoutrements of a tanker-size corporation?

Finally, there is the galloping expansion of global organisations or hugely rich American independent companies that are sweeping through the television world like a bushfire. Sky, HBO, Amazon, Netflix – on they go, sometimes doing what the BBC has previously done and with no loss of excellence.

Where does that leave the BBC? There are enemies circling the wagon train, but it also has an enormous number of supporters. Its independence is deeply valued. It still has muscle. Its freedom from advertising is greatly appreciated by many listeners and viewers. Radio 4 is the biggest commissioner of drama in the world. There are the unique Proms. There is its association with the Open University. There are regular instances of brilliance in its drama, documentaries and comedy. It still does programmes that no other broadcaster would attempt to do. People in this country trust it more than they trust or listen to any political party or organisation.

And that, it seems to me, is its salvation. The BBC seems too cowed by its enemies and too shy to summon up its friends. It has many groups who will fight for it. Some of them set their clocks by it, others feed their minds and imaginations on it. There must be a way for the BBC to mobilise these groups and make its audiences a factor in the struggle it now has.

The BBC in the end is the sum of its programmes. Content will be key as channels proliferate. The BBC has proved over decades that it can deliver radio and television programmes across the waterfront that are deeply respected and cherished. It is now widely thought that it is time for the BBC to slim down for the fight, reach out to those it has served for so long and let them be part of its case for long-term survival, not just for a few more years but for as long as our few other deeply rooted institutions. It is a glorious British invention and has become integral to our sense of ourselves.

There’s no insurmountable reason for the new noise to be turned down or for the ghost of Reith to be exorcised. But the BBC itself must lead the charge against its critics.

• This New Noise by Charlotte Higgins (Guardian Faber, £12.99). To order a copy for £10, go to bookshop.theguardian.com or call 0330 333 6846. Free UK p&p over £10, online orders only. Phone orders min. p&p of £1.99.