For those of us of a certain age (62, since you ask), the extraordinary events of 5 May 1980 will remain indelibly etched on our memories. No, I’m not talking about the final of the Embassy World Snooker Championship, but the SAS’s storming of the Iranian embassy in London that suddenly interrupted the BBC’s coverage of that battle in Sheffield.

Few people had heard of the SAS (Special Air Service) before they explosively ended the six-day siege by six gunmen who had taken captive 26 hostages, including four Britons. But on that bank holiday evening on live TV a transfixed nation watched these mysterious figures dressed in black and wearing balaclavas smash their way into the embassy in Princes Gate.

In the space of a few action-packed minutes the special forces regiment went from obscurity to global renown, by liberating all but one of the surviving hostages – one had already been shot dead by the captors, and another died during the assault – and killing five of the six gunmen. After the ensuing celebrations, what had actually taken place in that grand stuccoed building overlooking Hyde Park was subject to conflicting testaments, government secrecy, myth-making, and a host of other confounding factors.

There have been plenty of books written in the aftermath and intervening years, most claiming to be the “true story”, but none as exhaustive or gripping as Ben Macintyre’s The Siege. Macintrye, who has written a number of bestsellers on espionage and the special forces, is a seasoned documenter of the British establishment’s cloaked histories. His retelling of Kim Philby’s betrayal – A Spy Among Friends – is one of the most enthralling accounts of a double agent’s ruthless deception.

That book was turned into a first-rate TV drama, with Damian Lewis and Guy Pearce, a previous book, Operation Mincemeat, was made into a feature film and a musical, and this latest will be adapted by the team behind Slow Horses. Here, he brings the same deft storyteller’s skill to what is a more complex and thought-provoking narrative than the one popularised by the siege’s dramatic conclusion. First of all the gunmen had an authentic cause to which they wanted to bring attention – the Arab population of Khuzestan in south-west Iran had been brutally persecuted by the Ayatollah Khomeini’s hardline Islamist regime that had taken control the previous year.

But they had been financed and deceived by the Iraqi secret service, who provided weapons for the mission, and the terrorist Abu Nidal, who planned it. There was also a wider context, the 53 American embassy staff held hostage in Tehran by students loyal to Khomeini. Six days before the gunmen entered Princes Gate, an American attempt to rescue its citizens ended in disaster when a helicopter had collided with a military transport plane in the Iranian desert.

Inevitably, and falsely, the Iranians denounced the hostage-takers in South Kensington as CIA puppets. Few in the British government had heard of – or cared much about – Khuzestan. And from the outset Margaret Thatcher was adamant that no concessions be made – nor would she allow the gunmen safe passage out of the country.

Apprised of the prime minister’s bottom line, the police played a waiting game, while preventing the media from airing the hostage-takers’ case. Macintyre conveys the building tensions with almost claustrophobic intensity, as unlikely bonding and volatile mood swings affected hostages and gunmen alike, and periods of peaceful solidarity gave way to terrifying death threats.

The hostage-takers had been told by their handlers that the siege would last no more than 24 hours before the UK government would capitulate. This was always transparent fantasy, not least because the demands included the release of political prisoners in Iran – a country that, as its pious foreign minister made clear, was only too happy to see its embassy staff martyred.

As each day passed, the gunmen became increasingly convinced that the police, who were drilling the adjacent walls to plant listening devices, were going to come crashing in at any moment. The paranoia that ensued helped ensure those fears were realised. The team began splintering under the relentless pressure, with some wanting to surrender while the others preferred to die. The division led to the shooting of the staunch Khomeini-supporting Iranian press officer that triggered the SAS’s intervention.

Macintyre unearths some gems of social history. When the SAS were about to go in, ITV producers refused to transfer its coverage to the embassy until after Coronation Street had come to an end. Yet at least one aspect of the culture has remained unchanged – the unwelcome presence of Prince Andrew. He unsuccessfully tried to stage a vainglorious visit to the siege scene while the police were still preoccupied with getting the hostages released.

What took place in the chaos of the SAS assault may never be definitively known, although Macintyre puts together as accurate a picture as possible. The big question will always remain whether the gunmen surrendered and were then killed by the SAS. As it’s impossible to imagine what it must be like to confront fanatics armed with machine guns, it’s unwise to make armchair judgments on appropriate conduct. That said, it seems likely that the soldiers were not looking to take prisoners. Indeed the only gunman to survive disguised himself as a hostage, and even then came close to being shot.

As a propaganda exercise for the Iranian Arab cause the siege was a total failure. The one surviving gunman (who’d wanted to surrender all along) served 27 years in prison. The ordeal also haunted the hostages, although PC Trevor Lock emerged as an understated hero. Assigned to guard the embassy, he managed to conceal his handgun for six days by keeping his heavy coat on and avoiding washing. He maintained a calm exterior right up until the moment he wrestled with one of the gunmen in the midst of the rescue operation.

Iraq and Iran were soon entrenched in warfare, and the Iranian foreign minister who had been content to see his staff murdered was himself tortured and executed by the regime he fervently supported. The only real winners were the SAS and Thatcher, who rode a wave of national pride all the way up to and beyond the Falklands war.

• The Siege: The Remarkable Story of the Greatest SAS Hostage Drama by Ben Macintyre is published by Viking (£25). To support the Guardian and Observer order your copy at guardianbookshop.com. Delivery charges may apply.

• The caption on the secondary image accompanying this article was amended on 19 September 2024. It shows a police officer, a hostage and one of the terrorists – not two terrorists as initially stated.