He was described by Graham Greene as the greatest journalist of the 20th century and attacked by senator Joseph McCarthy as “one of the most dangerous ‘reds’ in the world”. In the 1930s he founded the Week, which exposed Nazi sympathisers in Britain, and interviewed Al Capone for the Times. Now the remarkable life of Claud Cockburn, who died in 1981, is being told by his son, Patrick, in a book which hails him as the inventor of “guerrilla journalism”.

Having been a rising star at the Times in the 1920s and early 1930s as a correspondent in Berlin and New York, Cockburn, the son of a British diplomat, foresaw what was happening in Germany as Hitler and the Nazis advanced and decided that the best way to combat this was through a newsletter – the Week. This would both publicise what was happening in Germany and entertain his readers with political gossip and insight. An article on what he would call the “Cliveden set”, about those in Britain who sought to appease Hitler, alerted people to what was happening in a way the mainstream press did not. The then German ambassador to Britain, Joachim von Ribbentrop – later to be executed at Nuremberg – complained angrily about the coverage.

In 1933, Ramsay MacDonald, whose plans to end the great depression by calling a world economic conference in London were mocked by Cockburn, was so enraged that he called an impromptu press conference to denounce him, holding up a copy of the Week. This propelled its name and reputation around the world.

What prompted the book now, more than 40 years after Claud’s death? “A few years ago I applied to MI5 for his files and about a year later I got 24 volumes of them,” said Patrick Cockburn, himself an award-winning correspondent for the i, the Independent and the Financial Times. “I had always wondered how effective the Week was and that kind of journalism he also did later, in the 1950s, for Punch and in the 1960s for Private Eye.

“What I learned from the files was that it had had an enormous impact. There were splenetic memos from senior officials, outraged at his disclosures and speculation about his sources and how he seemed to know exactly what had happened in a cabinet meeting two days earlier. There was some of this stuff in his 1967 autobiography [I, Claud...] but an awful lot he hadn’t written about.”

The cast of characters Cockburn engaged with over the years is striking. In 1930, he went to Chicago for the Times to interview Capone, who gave him a lecture on the virtues of American capitalism. The writer Christopher Isherwood was a fan. “I find myself warming to Cockburn,” he wrote in 1934. “I get the Week regularly… he does slash out at these crooks and murderers and he’s so inexhaustibly cocky and funny, like a street-boy throwing stones at pompous windows.” At lunch with Charles de Gaulle in Algiers in 1943, the general asked Cockburn why he was a communist. After listening to his response, de Gaulle asked: “Don’t you think your view is somewhat romantic?”

Cockburn had joined the Communist party in the 1930s and wrote for the Daily Worker (now the Morning Star). He covered the Spanish civil war for the paper and, as Patrick Cockburn puts it, “he thought the communists alone had the fanaticism and discipline to give the republic a fighting chance against General Franco”. Reporting alongside him was the redoubtable Jean Ross, the second of his three partners, who was covering the war for the Daily Express and would file copy under Cockburn’s name for the Daily Worker when he was at the front. Isherwood based Sally Bowles in Goodbye to Berlin on Ross.

In Spain, he not only fought with the republican militia but engaged in an elaborate hoax to help it. Field guns for the republican military were being held at the French-Spanish frontier but Cockburn and his friend Otto Katz felt that if the French prime minister Léon Blum was to learn of a mutiny against Franco in Spanish Morocco he might let the guns through. A bogus story about the “mutiny” duly appeared and might well have led to Blum relaxing the border controls.

When Cockburn told the story of the fake uprising a couple of decades later, it “did him reputational damage”, as Patrick Cockburn puts it, but his father was unrepentant.

“I felt that Claud’s derisive but unwavering suspicion of governments and the mainline media, always expecting the worst from both, was the correct posture for a polemical journalist,” wrote Patrick in the introduction. “He disbelieved strongly in the axiom about telling ‘truth to power’, knowing that the rulers of the Earth have no wish to hear any such thing. Much more effective, he believed, is to tell truth to the powerless strongly, so they have a fighting chance in any struggle against the big battalions.”

Was Patrick Cockburn surprised by what he discovered in his research? “Claud was a much tougher and more practical man than I’d pictured him growing up. He thought very carefully about how a single journalist, without resources, could influence events and how could a non-professional army win a war. A further reason for me deciding to write the book was is that, over the years reporting crises and wars, I came to feel that Claud’s scepticism about the competence and good intentions of governments and powerful organisations in general was fully justified – something glaringly true today.”

Claud Cockburn’s three sons all became journalists: Alexander, who died in 2012, moved to the US and wrote for Village Voice, the Nation and CounterPunch; Andrew is the Washington editor of Harper’s and I have known them all.

With the growth of the far right in Europe, and conflicts across the world, Patrick Cockburn said his father’s work was particularly relevant. “These crises that turn into wars and governments increasingly having to cover things up means that dissident journalism is still relevant. It’s as important now as it was in the 1930s. David can still do serious damage to Goliath, when you could wage an information guerrilla war against the powers that be.”

• Believe Nothing Until It Is Officially Denied: Claud Cockburn and the Invention of Guerrilla Journalism by Patrick Cockburn is published by Verso on 22 October (£30). To support the Guardian and Observer order your copy at guardianbookshop.com. Delivery charges may apply