This is the compelling story of how one intensely motivated man grabbed control of a broken party, eviscerated its left and ruthlessly reforged Labour into a power-hungry machine. The central character of the plot is not Keir Starmer; the main protagonist is the operative and strategist Morgan McSweeney. The Irishman will be unknown to the great majority of voters, but it is the claim of this account that he has been one of the most consequential figures in contemporary politics.

It becomes clear early on that Starmer will often be peripheral to the tale of how Labour was taken from abject defeat in 2019 to victory with a parliamentary landslide in 2024. The Labour leader is largely absent from the first 40 or so pages. Everything is McSweeney and McSweeney is everything, to use the kind of formulation that these authors are rather fond of. The school-hating son of an accountant and a clerical worker, he comes from the small market town of Macroom in County Cork. In search of wider horizons, he makes his way to London as a “17-year-old slacker”. I assume this is a self-description that he offered to the writers to whom he’s obviously given a lot of cooperation. He gets work in the capital as a labourer on building sites. Tiring of it, he tries university, only to drop out within 12 months. The authors suggest that the making of him is three months spent in Israel on a kibbutz, where “the lazy teenager learned to work”. He takes a degree in politics and marketing at Middlesex University.

Mesmerised by the early triumphs of Tony Blair, he secures menial roles on the campaign operation built by Peter Mandelson. Journalists sometimes ascribe his rise to being a Mandelson protege, but the authors suggest that none of the luminaries of New Labour had a clue who the Irishman was back then. He cuts his campaigning teeth in Lambeth, south London, where Labour’s reputation has been poisoned by the hard left, and then in Dagenham, east London, where it is a bare-knuckle fight with the far-right racists of the British National party. It is these experiences on some of the meaner streets of the capital that formed his muscular and sometimes fanatical approach to politics, and fed his limitless disdain for leftwing activists and politicians who obsess over their pet ideological preoccupations rather than speaking to and catering for the concerns and needs of the voters. It also impressed upon him the importance of data and messaging.

He’s a cunning fellow, this red-haired Irishman who speaks extremely fast in a very soft voice. When Jeremy Corbyn and his comrades gain control of the party in 2015, McSweeney responds with an organisation called Labour Together. Rather than be openly oppositional with all the risks that would entail, he goes to Corbyn to present the outfit as benignly helpful to his leadership. This is a confidence trick. “In his mind, Corbyn’s politics were not just wrong. They were evil.” His real purpose is to use any means necessary to destabilise and delegitimise Corbynism and Corbynite organisations, the better to be ready to destroy them altogether when the time is ripe.

As Corbyn begins to spiral towards catastrophic defeat, what McSweeney lacks is “The Candidate” to replace him. He alights on Starmer. Only a few others had spotted the strength of the ambition throbbing beneath the north London lawyer’s vaguely progressive and rather bland exterior. We’re told that he started coveting the job of leader at least five years before he got it and sought to improve his public performances by taking acting lessons from Leonie Mellinger, who once appeared alongside Rik Mayall in The New Statesman. Rachel Reeves and other leading lights were conscientious objectors to Corbynism and refused to serve in his shadow cabinet. That disqualified them in the eyes of the leftwing membership, but Starmer could attract support because he had served – teeth gritted, but more or less loyally – on Corbyn’s frontbench. Starmer does not have a lot of his own politics, but this is an advantage for McSweeney because he has spent a huge amount of a rich donor’s money on polling the party’s membership to find out what kind of candidate they’d back.

A picture begins to build of Starmer as not the master of his own fate, but someone acting a part others have ghostwritten for him. The Corbynesque policy pledges that he issued to secure the leadership were not his own work. They were scripted for him. We also learn that much of his first speech as Labour leader was written by a McSweeney associate whom Starmer had never even met. Labour strategy is not developed in the shadow cabinet. The plan to reconnect Labour with the working-class voters it has lost was called Project Phoenix and discussed at McSweeney’s kitchen cabinet of like-minded Labour apparatchiks who meet in secret on Sunday evenings at the London home of Roger Liddle, a Labour peer and old friend of Mandelson.

Starmer begins his leadership thinking his guiding philosophy should be unity, as he’d promised when getting elected. McSweeney regards unity as a euphemism for cowardice and thinks there cannot and should not be unity with the left. “Let’s all be friends” is the strategist’s rather derogatory take on Starmer’s initial desire to be a conciliator of the factions. The inflection point is Labour’s grim defeat at the Hartlepool byelection, characterised by a senior Labour official as “a fucking shit show in the fuck factory”. Boris Johnson celebrates by parading around the constituency accompanied by a 30ft inflatable effigy of himself. It is not a revelation to me that Starmer responded to this humiliation with a massive wobble. I was aware that he’d told intimates that he was going to resign, but didn’t know exactly how low he sunk. One of the merits of an extremely well-sourced account is the dramatic detail that the authors provide about the sliding-doors moments. They present us with a dour and disconsolate Starmer damning himself: “I think I have to go… We’re going backwards. We’re losing seats that Corbyn won.”

By contrast, McSweeney regards the drubbing in Hartlepool as “the kick up the arse the party needed”. There follows the threat of a coup by Angela Rayner and a near-death experience for Starmer at the Batley and Spen byelection. McSweeney begins to pursue the “cleansing confrontation” with the left that he had longed for to change Labour and to demonstrate to the country that it had changed. He successfully plots to ambush the 2021 conference with a rewrite of the party’s rulebook designed to forever emasculate the Corbynites. No one outside his tight circle of collaborators is told what he is up to until the very last moment. Starmer is one of those excluded from McSweeney’s need-to-know list. The transformation that follows is executed with such remorselessness that a year later the conference marks the death of the queen by singing the national anthem without a peep of dissent.

Johnson is brought down, Liz Truss blows up and Rishi Sunak can’t keep his head above water while McSweeney refines an election strategy that is designed to make the Labour vote as “efficient” as possible by denying resources to seats that will never be lost or cannot be won in order to concentrate effort on the swingers. Strategy is controlled by him and Pat McFadden, a veteran of the Blair years, from a windowless room at Labour HQ that is nicknamed “The Cell”. On this harsh account, Starmer’s contribution to the campaign amounts to fumbling the first TV debate, presenting “the threadbare policy offer” contained in the manifesto and lucking out when Sunak makes the suicidal decision to leave the D-day commemorations early.

McSweeney should try to keep this book away from the Labour leader, because he will surely loathe it. One of many spicy quotes that juice up the narrative has an adviser saying: “It’s impossible to work out whether Keir realises he is a pawn in a chess game. Or does he like being a pawn in a chess game, provided it makes him powerful.” There are many occasions when Starmer is depicted as placing himself “at a degree’s remove from the business of his own future”, subcontracting big calls to others and being opaque and detached when decisions have to be made or warring colleagues need sorting out. McSweeney is quoted confiding to a friend: “Keir acts like an HR manager, not a leader.” One of the strategist’s acolytes likens Starmer to a useful idiot with a reference to the driverless, remote-controlled trains on London’s Docklands Light Railway (DLR). “Keir’s not driving the train. He thinks he’s driving the train, but we’ve sat him in front of the DLR.” Recruits to the operation from the New Labour years had worked for leaders who wrote their own electoral destinies. They find Starmer weirdly disengaged. “We were used to, in our Blair-Brown meetings, watching Tony and Gordon thrash out the strategy. But, literally, Keir was like: ‘Right, you guys get on with it.’”

Though it was evident that Labour was on its way to power well in advance, preparations were thin. Nick Boles, a former Conservative minister and chum of Michael Gove, is astonished to find himself helping to draw up plans for the first Labour government in well over a decade. “There I was: 58 years old, a failed politician, and I was producing all the organograms on PowerPoint. Fuck knows – I literally must be the least well-qualified person to do PowerPoint slides.”

Poor old Sue Gray, the civil servant whom Starmer chose to be his chief of staff, is the target of yet another dumper truck of opprobrium. The authors’ hostile sources depict her as a power-hoarding, accessing-denying control freak whose preparations for government existed only in her head. One of her mistakes was to threaten the job security of the male Starmer aides whom she privately derided as a lazy, louche and badly dressed “boys’ club”. The fatal error was to cross McSweeney. In a power struggle between the two, a recent recruit from the civil service was always going to lose to the Irishman who had made himself so indispensable. She got the boot; he got her job.

McSweeney is the hero – or antihero, according to taste – of a rattling tale terrifically well told. He might be flattered, but he should also be worried. Machiavellis attract resentment. The more so when they stray out of the shadows and into the limelight. Prime ministers don’t take well to being portrayed as the dumb instrument of someone else’s designs. This book will do something to McSweeney’s relationship with Starmer – and I doubt it is going to be anything good.

• Andrew Rawnsley is the Chief Political Commentator of the Observer

• Get In: The Inside Story of Labour Under Starmer by Patrick Maguire and Gabriel Pogrund is published by Bodley Head (£25). To support the Guardian and Observer order your copy at guardianbookshop.com. Delivery charges may apply