Even a brilliant newspaper editor can undersell a good story on the front page. At the start of Lionel Barber’s account of the world from his perspective as editor of the Financial Times between 2005 and 2020, there is an extensive list of dramatis personae, almost all of them male. (“One day,” he writes later, “I will deal with the alpha male problem, but not today.”) The players are broken down into their categories: politics, business and finance, royalty, journalism and diplomacy. The heart sinks – if journalism sits so easily in such a cast list, how can it do its job of comforting the afflicted and afflicting the comfortable?

When, in the preface, Barber writes of being “granted audiences with royalty such as Prince Charles, Prince Andrew and Mohammed bin Salman”, the sinking heart becomes the creeping flesh. The idea of an editor being “granted an audience” is bad enough. When one of those gracious enough to allow him into his presence is an alleged killer and another pals around with a convicted sex offender, it threatens to put the sick into sycophancy.

As an advertisement for himself, in other words, Barber’s publication of his diaries in what seems a heavily edited and revised form, is, for the first 100 pages or so, rather counter-productive. A reader uninterested in the newspaper business would not guess that this is the man who raised the FT’s journalistic standards, made it a global brand, navigated its transition into the digital world, attracted a million paying readers and steered it through a takeover by the Japanese media group Nikkei.

Barber quotes a Daily Mail description of him as a “weapons-grade social climber and name-dropper extraordinaire”, and at times he seems intent on living down to this reputation. He is capable of sentences that Craig Brown could steal wholesale for a Private Eye parody diary. Encountering the former Sun editor, Rebekah Brooks, he reminds us that “We’re now on first-name terms after our pre-Christmas lunch at San Lorenzo, Princess Diana’s favourite celebrity hangout.”

When, in 2011, his friend Mario Draghi becomes president of the European Central Bank while the euro is on the brink of collapse, Barber notes: “We were in this crisis together.” When there is a problem requiring his attention at the FT he sighs that “and of all places I’m in Birmingham”. His readers will feel the sheer, desolate isolation from the real world of what he repeatedly and without irony calls the “movers and shakers”, in which titans such as Sir Fred Goodwin, the genius who would drive RBS almost to extinction, drop in for lunch to share their wisdom.

And yet there is a good story here. There are some telling vignettes. In 2008, after the FT has revealed that Dubai is bankrupt and is being bailed out by Abu Dhabi, Prince Andrew calls Barber to lean on him. The dialogue is a tiny masterpiece in which Pinteresque menace is undercut by Coco the Clownish idiocy. HRH: “Your man in Dubai, Simon Carr, is causing a lot of trouble.” LB: “You mean Simeon Kerr.” HRH: “Yes, Simon Kerr … Look I’m just passing on a message. Your man is causing a lot of problems.”

In the same summer of 2008, the then Democratic candidate for the presidency Barack Obama visits London and meets three political leaders. His laconic judgments are relayed to Barber: “On Blair: sizzle and substance. On Brown: substance. On Cameron: sizzle.”

And yet Barber goes with the sizzler. He shifted the FT’s editorial line from being broadly Blairite to reliably supportive of David Cameron, a man he subsequently describes as “really a PR merchant with no hard political core”. The true substance of this book is his disillusionment, not just with the PR merchant, but with the feral capitalism that created the banking crash, pointless austerity and a profound crisis of democracy.

To his great credit, Barber is sufficiently self-aware to understand that, although he and the FT did not create the monster of reactionary populism, their cheerleading for austerity helped to feed it. He really begins to see its consequences when, during the Brexit referendum of 2016, Vote Leave’s mastermind, Dominic Cummings, visits the FT for lunch. Asked about the scale of the economic shock that would follow Brexit, “Cummings says he has no idea what damage such a shock might do. The figure is unknowable and the ordinary person in the street would have little understanding anyway.” As he realises what he has said, Cummings adds: “This conversation is off the record, right?”

It shouldn’t be. The drama inherent in Barber’s position lies in the irresolvable tension between closeness to power and journalistic objectivity. Capitalism, as practised for most of Barber’s tenure at the head of its house journal, is an exclusive game, a system run by and for a tiny elite. He does not disguise his notion of his mission as editor of the FT: to become as intimate as possible with – as he also repeatedly calls them – the “power players”. But he is also an excellent journalist, committed to serious and independent reporting.

The dilemma is summed up in an anecdote about how the FT is trying to prove that the neocon Paul Wolfowitz, as president of the World Bank, improperly ordered a salary rise for an employee with whom he had formed a personal relationship. Barber is at a party in Christopher Hitchens’s sumptuous apartment in Washington and announces: “I want that story nailed – and Wolfowitz too.” Unbeknownst to Barber, Wolfowitz is standing right beside him.

When John Browne, the chief executive of BP is forced to resign because of his relationship with a male escort, Barber calls him: “I reassure him that his role as the outstanding businessman of his age stands (though I skirt over BP’s safety record in the US).” It is hard not to get whiplash from the recoil of that parenthesis and the cognitive dissonance created by such flattery. And yet, the FT under Barber actually did superb work on BP’s disregard for safety – just as it did on a wide range of stories that exposed members of the business elite, most recently in portraying the huge Wirecard financial services operation as essentially fraudulent.

The outcome of the tension between the need to suck up to the very rich and the imperative to expose their depredations was that Barber’s FT movie had great closeups and very blurry wide shots. Shocking individual exposés did not lead to sufficient scrutiny of the big picture. Gradually, Barber acknowledges all the things he did not see: “Failing to appreciate popular disenchantment with authority in the wake of the financial crisis”; “We could have better flagged the risks when HBOS and Northern Rock made their dash for growth in the credit boom”; “Maybe we were too close to the top people in the City and on Wall Street”; “We were slow to understand and report on the level of tax arbitrage among footloose multinationals”; “We should have debated more thoroughly the costs of austerity”.

That’s quite a lot to miss. It happened, not because Barber or the fine journalists who worked for him were stupid, but because there is a tacit contract at work. If you want to be intimate with the power players, you can reveal some of their dirty secrets. What you must not reveal is what is actually not a secret at all: that the world they have created is neither sustainable nor compatible with liberal democracy.

• The Powerful and the Damned: Private Diaries in Turbulent Times is published by WH Allen (£25). To order a copy go to guardianbookshop.com. Delivery charges may apply.