Critics were hard on Stephen Spender, but not as hard as he was on himself. He felt a failure, a man famous only for having famous friends. Each time he published something new, it was like throwing himself to the wolves. "People I meet sometimes say 'I've read your books.' Secretly, I do not believe that anyone has read anything (apart from a few anthologised poems)." Even achievements he might have been proud of brought him little satisfaction: "My lecture an enormous success (by this I do not mean it was a good lecture)." Accompanying everything there was a voice that said "You are wrong."

In a long career, he took up many noble causes – for peace, democracy, freedom of expression, intellectual exchange. But despite, or because of, his missionary goodwill, he tended to be seen as a duffer – shy, awkward, naive. "You will always be a poet because you will always be humiliated," Auden said soon after they met, and humiliation came naturally to him. The scandal over the CIA funding of the magazine he co-edited, Encounter, was his most public humiliation. His journals also include more comical examples, such as his account of walking down a street in Covent Garden and unwittingly, noisily farting, to the delight of some nearby boys and girls. He's more amused than embarrassed by this, until a "self-important" thought strikes him: what if they'd realised it was Stephen Spender farting? What would they think then?

Such agonised self-consciousness is an endearing feature of these journals, spanning more than 50 years and including personal material that hasn't appeared before. This isn't to say that they're confessional: on the contrary, when asked whether he was totally candid in his journals, Spender replied that he "did not feel impelled to be – or rather, I felt impelled not to be". Short on bitchiness, and with half an eye on publication, he's too nice to be a great diarist. But he's unfailingly curious and liberal-minded – and so down on himself that you can't help but take his side.

A close friend once put it to him that the reason he'd lost his way as a poet was that "you haven't been able to deal with the problem of your homosexuality in your writing." The friend, Reynolds Price, was also a lover, though you would never know that from the journals, which rather proves the point. Spender defends himself on the grounds that writing about his homosexuality would have resulted in sentimentality and "a lack of contact with the ordinary life of family marriage". More to the point, he wanted to spare his wife Natasha, whose sensitivity about his public image was greater than his own.

Were Natasha alive (she died two years ago), she would certainly have censored the most intimate passage in this book, which describes Spender's relationship with an American biology student almost 50 years his junior, Bryan Obst. Nothing explicit is recorded, in the manner of Edmund White or Alan Hollinghurst (neither of whom Spender seems to have read). But the pleasure he takes in his young lover is palpable. And there is one tempestuous scene, when Natasha overhears or intuits a furtive phone call he's making to Obst from the next room and he comes through to find her crying. The relationship continued, over odd snatched weeks, till Obst's death from Aids. But Spender denied or underplayed it, as he did his affairs – past and present – with other young men.

Natasha's preoccupation with her (and his) reputation is a recurrent theme in the journals, notably when a biography appears alleging that she had an affair with Raymond Chandler – distressed and indignant, she takes to her bed. Spender's response is sweetly supportive: even if true, it's irrelevant, he tells her; whatever she did for Chandler was done from compassion. A "bad conscience" about his own affairs made it easier for him to be understanding, perhaps. But his concern for her was genuine. He couldn't imagine life without her. The marriage wasn't just a front.

His absorption in "family marriage" is also evident in the many entries here about his two children, Matthew and Lizzie. He and Matthew regularly go off on "honeymoons" – bonding trips abroad – and he's equally close to Lizzie: they watch Some Like It Hot together on TV, which "really cheered us up after a dreary day". Even when the children have grown up and are doing well, he still worries about them: "One doesn't have reserves of philosophical – or cynical – indifference to one's children. Their unhappiness seems unbearable. It is worse than one's own unhappiness."

If Spender the devoted father comes as a surprise, Spender the dinner guest and lunch host is more familiar – but mightily impressive nonetheless. Can any other 20th-century writer claim to have hobnobbed with all of the following: Igor Stravinsky, Charlie Chaplin, Princess Margaret, Truman Capote, Ted Heath, Yehudi Menuhin, Guy Burgess, Sarah Ferguson, Hugh Gaitskell, Michel Foucault, Jackie Onassis, Margaret Thatcher, Susan Sontag, Beth Chatto, Vivien Leigh, Francis Bacon, Tony Benn? It's typical that while visiting the Guggenheim Museum in Venice, Spender is spotted among the tourists by Peggy Guggenheim and invited to cocktails. When he analyses the work of the artists and musicians he knew, he's not insightful or deep. But they're charmed by him and he by them. Whatever his other talents, Spender was good at making (and keeping) friends.

The endless lunches and lecture tours took their toll on his poetry, which came slowly or not at all. The journals were a fallback – a record of interesting times and interesting people that would guarantee him a place in posterity – but, as Spender was the first to admit, they were no substitute for the masterpiece he'd hoped to write before he died: "Under it all, there is the feeling I have never done my best." The later journal entries, some written during spells in hospital, are much possessed by death, without being terrified of it: with so many friends having predeceased him, he feels more at home among the dead than the living. His chief worry is how his death will inconvenience those left behind.

Among those who died before Spender was Auden, the dominant presence in the journals and a constant point of comparison. Spender envied his poetic gift but noted that Auden likewise envied him, first for having a large penis ("He was certainly affected by this and mentioned it on many occasions") and second for being a father. He admired Auden's devotion to work but was also pulled in the opposite direction, towards sociability, hedonism, "travelling first-class, giving people delicious meals, etc". When the choice came – perfection of the life or the art? – he went for life. And on balance, for all his doubts and sense of failure, and despite the voice telling him "You are wrong", he believed he'd chosen right.