In Britain, the army is one of the few institutions it is almost impossible to subject to serious criticism without provoking outrage. One reason is that anyone who does raise the possibility that it is not world-beating risks accusations of being unpatriotic and disrespectful towards the brave men and women who endanger their lives to keep us safe.

Another reason, which author Simon Akam explores in great detail in his excellent and valuable book, is that the British army has made great and often very effective efforts to close down any criticism, constructive or otherwise. The publication of the book was fraught with difficulty as the military establishment closed ranks. (The original publisher was Penguin Random House, which put the book on hold, telling Akam there was a “quite unprecedented level of withdrawal of support and co-operation for the book from multiple sources”). This alone goes a long way to substantiating the author’s powerful accusation that the army, a reflexively defensive, instinctively conservative and opaque institution, has limited ability to adapt to change, whether military, social or political. And this means things go wrong.

Not all of what went so very badly wrong in Iraq and Afghanistan can be conveniently blamed on the politicians or on penny-pinching Treasury mandarins. As Akam shows, senior officers made grave errors of judgment. Some may have been traumatised or exhausted, but others may simply have not been particularly competent. There is much material here about the personal rivalries between the relatively small number of senior soldiers in Britain’s relatively small army.

The book’s dense, detailed narrative opens in 2002 with a veteran specialist giving a virtuoso display of armoured warfare on a Canadian training ground. He receives fulsome praise from his superior officers even though this type of combat is unsuited to the wars his comrades-in-arms are about to fight – and in spite of the Nazi officer’s uniform he is found to be wearing underneath his British one.

But this is apparently fine in the British army, back at the dawn of the new millennium. Akam takes us into the barracks and bases in Germany on the eve of the Iraq war. There are posh officers who will only drink Pol Roger champagne, beery “drink until you die” sessions for other ranks, appalling rituals to humiliate new recruits, an inflexible hierarchy, snobbery and almost no one who has actually ever fought. Akam points out that, unlike almost every other profession, soldiers can spend decades training without actually doing what they are trained for: combat.

And so to the debacle of the British army in Iraq, which came after years of senior soldiers telling journalists, politicians and everyone else that they were emphatically not like the Americans, because the UK had seen low-intensity conflicts in Northern Ireland and Yemen and Malaya and so understood how to “win hearts and minds”. When I was reporting on the conflict I accompanied patrols in the southern Iraqi city of Basra, given to the British by US planners to look after while they headed north to Baghdad. The soldiers wore berets, not helmets, and were led by sergeants bellowing “Salaam alaikum” in strong regional accents at disconcerted local residents.

I drove into a small town called Majar al Kabir where, just months after the 2003 invasion, six British military policemen on a training mission had been lynched. I heard what people had to say about the murders, and it became clear that the British army’s confidence in its supposed know-how was utterly misguided. In the end, the US had to come in. Akam has also done a thorough job on the alleged abuses committed by British troops during the conflict.

In Afghanistan, from 2006, a host of further deficiencies were laid bare. There were never enough troops, nor helicopters, nor the right equipment. A policy of six-month rotations for units promoted rivalry and dramatic discontinuities. Command chains were impossibly complex. “War porn” shot on soldiers’ smartphones during combat was not just tolerated but actively disseminated by officers, while dozens of books full of stories of battlefront derring-do were supported by senior officers. This culture of video game violence and a frenzied chase after medals all made a pivot to a less “kinetic” strategy very hard to execute when commanders decided that a change of approach was necessary. In Afghanistan, as in Iraq, the US military came in to finish jobs the British had been unable to complete. When the wars in Iraq and Afghanistan ended, there were big public inquiries that received lots of attention but almost no serious scrutiny of or sanction for senior soldiers.

This is a long book. There are chapters of useful and rigorous investigation of alleged abuses committed by British troops in both theatres, and these may have explained the ire of some interviewees. That Akam has taken the trouble to interview sex workers about the scared British soldiers who come to them to talk as often as to have sex on the eve of the Iraq war is impressive. The detail often makes for gripping individual episodes, but sometimes clouds the overall argument and narrative.

It would be unfair to expect more analysis or reporting of the broader context of British intervention in Iraq or Afghanistan in a work that took five years to write. But without it, the most important factor in success or failure – local and regional politics – is underplayed. In both wars, the self-appointed “best little army in the world” was only a minor actor, and one that had a limited impact. This is another bitter truth that many senior soldiers have trouble accepting. In this at least, the army is a representative of the country it fights for.

• The Changing of the Guard: The British Army Since 9/11 is published by Scribe (£25). To buy a copy go to guardianbookshop.com. Delivery charges may apply.