One of the darkest and most enthralling British espionage stories of the 20th century turns 50 this month, still resonant with sinister meaning. It was on 1 July 1963 that the British government finally admitted what it had known for some time: that Harold Adrian Russell Philby – "Kim" to friends and family – was not merely living in the Soviet Union as a defector and a Russian spy, but was actually the fabled "third man". Later this archetype of treachery would become known, in the words of his biographer, as "the spy who betrayed a generation".
Philby was perhaps the most lethal double agent in the annals of British espionage. As a member of the Cambridge Five spy ring and a secret servant of the Soviet intelligence services, Philby was responsible for the betrayal of countless national secrets as well as the brutal elimination of many British agents.
At the same time he was a member of the British establishment, with a distinguished literary father and friendships with prominent English literati, such as Graham Greene, as well as with high-flying US spooks such as James Angleton, later to become head of the CIA. If ever there was a member of the club – two, as it turned out – it was Kim Philby, OBE.
When his fellow spies Donald Maclean and Guy Burgess fled to the USSR in 1951, Philby's position seemed badly compromised, but after repeated interrogation he was exonerated. In 1955 foreign secretary Harold Macmillan told the Commons that he had "no reason to conclude that Mr Philby has betrayed the interests of his country, or to identify him with the so-called 'third man', if indeed there was one".
That exoneration has special meaning at this newspaper. From 1955 until his abrupt defection in 1963, Philby was the Observer's Middle East correspondent, based in Beirut. His editor was also a renowned figure in the 1950s – David Astor, who commissioned work from George Orwell, Arthur Koestler and Kenneth Tynan.
Their stories have become controversially braided together. Astor had fought with undercover units in France. As editor, he employed journalists with connections to British intelligence – Gavin Young and Mark Frankland – as well as wartime survivors such as Mark Arnold-Forster and Edward Crankshaw.
Out of this, questions about Astor's role in Philby's career have lingered like gunsmoke after a shooting. It is sometimes alleged, most recently on Radio 4's Document series, that Astor was in the pay of the secret service.
Jeremy Lewis, who is writing Astor's biography for Jonathan Cape, told the Observer that Astor's knowledge of the secret world was both less sensational and more complicated than hitherto understood. His actions reveal him as a man not afraid to take brave decisions on matters of profound democratic principle, as he was to demonstrate famously during the Suez crisis in 1956.
Astor was never a spy, says Lewis, but part of a wartime generation in which demarcation lines were blurred. Astor and Philby, both born in 1912, came from established but maverick families. Astor, second son of newspaper proprietor Waldorf Astor, was of Anglo-American stock. Philby was another kind of insider-outsider, the son of St John Philby, a well-known writer, orientalist and convert to Islam. Greene, eight years older than both men, would later become Philby's friend and champion. Greene wrote of his Edwardian contemporaries: "We were a generation brought up on adventure who had missed the enormous disillusion of the first world war, and so we went looking for adventure."
Philby went to Cambridge and was recruited, with Burgess and Maclean, as a Soviet mole. By contrast, Astor went to Oxford and mixed with anti-Nazi intellectuals such as Adam von Trott zu Solz, who was later executed for his part in the 1944 bomb plot to kill Hitler. In 1939, now flirting with journalism, Astor found the adventure he sought. That summer his cousin, the comedienne Joyce Grenfell, reported with excitement that "David is said to be doing secret service work". Who knows if that was true? Astor always maintained that he had been turned down by MI6. But then a much bigger adventure turned up, one that would engage everyone's attention: Hitler invaded Poland.
Soon after Neville Chamberlain's declaration of war in September 1939, MI6 began overtures to anti-Nazis in the German army in order to destroy the Nazis from within. By November, a secret rendezvous had been arranged to put a plot into action. "I have a hunch", said Chamberlain, when told of the operation, "that the war will be over by the spring."
Two British spies headed for Venlo on the Dutch-German border for a high-stakes meeting with the leader of the German opposition, a certain Hauptmann Schammel. But the operation was really being run by Himmler – the SS had staked out the rendezvous, and "Schammel" was actually a member of the Reich security service. The two hapless British agents were surrounded by SS plainclothes men and hustled off to Berlin for a Gestapo interrogation.
Astor had hovered on the fringes of the Venlo debacle. As a close friend of Adam von Trott zu Solz, he was familiar with the German opposition. In the autumn of 1939, he even went to Holland, posing as a Times reporter, and met the two future Venlo agents, but was not impressed.
The "Venlo Incident" was a humiliation for British intelligence. Churchill had followed the incident with dismay. Once Churchill assumed power in 1940, he put secret operations in occupied Europe on a war footing. The recruitment of thrill-seeking literati such as Ian Fleming, Dennis Wheatley, Malcolm Muggeridge and Greene placed covert operations at the psychological heart of the war effort, inspired by Churchill's love of adventure.
This corps d'elite was a group of writers and academics, notably Hugh Trevor-Roper, recruited from Oxbridge and the BBC. Some of them became intimately connected through shared girlfriends and mistresses – women such as Elizabeth Bowen and Sonia Brownell, later Orwell's second wife.
Lewis says: "During the war, through his friendship with Orwell and his work at the Observer, Astor was at the fulcrum of three immensely talented circles which all played a vital role in the war effort on the home front. There were social democrats like Richard Crossman and Michael Foot; refugees like Isaac Deutscher, Sebastian Haffner and EF (Fritz) Schumacher; Orwell and Arthur Koestler, both of whom were to become, in effect, keen Cold War warriors after 1945." Astor and his circle, meeting at the Shanghai club in Soho, were part of the non-communist left, progressive intellectuals (William Beveridge and Stafford Cripps) planning for a postwar New Jerusalem.
The process by which the life-and-death struggle with the Third Reich became the cold war with the Soviet Union was glacially slow. While the Foreign Office played along with Stalin and the Soviet Union, Philby established himself deep in Whitehall. He worked for the Special Operations Executive, had links with Bletchley Park, the codebreaking headquarters, and ended the war as deputy head of Section V – counter-intelligence.
Meanwhile, prewar habits of intelligence-gathering persisted. An extraordinary amount of British espionage business was still conducted in London's clubland. Here the spirit of John Buchan's Richard Hannay morphed into Ian Fleming's James Bond and, ultimately, John le Carré's George Smiley.
In the world of nods and winks in the clubs of Whitehall and St James's, the line between journalism and the defence of the realm was indistinct. Being familiar with members of the security services was not the same as being a spy. But it could mean some strange meetings. In 1952, after Burgess and Maclean had defected, Philby was left fighting to protect his position within the establishment. He applied to work at the Observer and met the news editor, Fred Tomlinson, Astor and the critic Philip Toynbee. Tomlinson, writing to a colleague, noted Philby's work for the Times and his engaging manner – "we all liked him" – but turned him down.
Lewis adds: "Although David Astor admitted to applying to join MI6 shortly before the outbreak of world war two, I've come across no evidence whatsoever in his archives that he was ever a secret service operative, or that he had any kind of formal relationship with MI6/SIS.
"As a newspaper editor, he was familiar with that world, including Kim Philby. I've no doubt some Observer journalists retained wartime connections with the secret services. But there's nothing to suggest Astor was a covert sympathiser, let alone an active agent."
When the cold war turned arctic in the early 1960s, a top interrogator, Nicholas Elliott, was sent to Beirut to confront Philby again . It was a cathartic meeting. On 23 January 1963, amid evidence of panic, Philby boarded a Soviet freighter and fled to Moscow. Six months later, on 1 July 1963, the knot of lies in his terrible story would start to unravel. The final phase of his "silent war" as a pathetic exile had begun.
* This article was amended on 2 August 2013 to correct the date David Astor travelled to Holland and to clarify a quote from Jeremy Lewis.
