Gordon Brown is a funny old stick: clever, complicated, full of contradictions. He’s got an autobiography coming out next week, My Life, Our Times, and edited highlights are already in circulation. There may be interviews to promote the book. Brown probably feels a bit uncomfortable about that aspect of writing a book. Brown is a man who often feels uncomfortable.

Brown probably feels uncomfortable about being the sort of politician who writes books at all. The money he makes from it will go to the Jennifer Brown Research Laboratory at Edinburgh University, set up in memory of the baby daughter he and Sarah Brown lost in 2002. Which is admirable, of course. One has the feeling that Brown would run a mile from any suggestion that he was “cashing in”, like other politicians.

Brown is a serious man, an intellectual. No one disputes that he has a brilliant mind. But I can’t think of many people I’d be less likely to turn to for insights into “our times”. Brown believes that his ministerial career ended with a lost election because he’s just not the sort of chap who fits in with our ghastly times.

“In a far more touchy-feely era, our leaders speak of public issues in intensely personal ways and assume they can win votes simply by telling their electors that they ‘feel their pain’. For me, being conspicuously demonstrative is uncomfortable,” he writes.

And, it’s true, our times are pretty ghastly. But Brown’s great failure was that he pandered to those times too much, not too little. Brown wishes now to assert himself as an enemy of neoliberalism. Yet he was in charge of finding a solution to Britain’s housing crisis in the 2000s, and he simply couldn’t find one that wouldn’t upset the short-term economic boom of the housing bubble, and, worse, make him look a bit socialist.

I’ll be interested to see what he says about that in his book. Not much, I’d imagine. It would be too painful to reflect too hard on that. Brown is less honest than he would like to be, especially with himself. His trouble is that he cares too much what other people think of him – excruciatingly much. It always has been.

Early on in the New Labour government, Brown was asked in an interview whether he was much of a user of that marvellous new thing, email. Rather than admit that this was not something he’s quite yet got to grips with, Brown blustered. Asked what his email address at the Treasury was, the silly git guessed, offering something that sounded a bit like an email address but wasn’t quite.

It was Brown’s own choice to be neither authentic nor sincere, because he couldn’t bear to admit even a small shortcoming. A similar impulse led him to invite the nation to picture him springing out of bed in the morning, to rock out to the Arctic Monkeys. Unable to name a single track on their debut album, he confessed to preferring Coldplay. Again and again, the poor guy tried to project himself as something he was not.

At a party once, when Brown was relaxed and had given a warm, funny speech, I cheekily shouted: “Nice speech, Gordon!” as I left. The guy turned towards the compliment with such a shy and grateful beam of delight on his face that I could have wept for him. The great Gordon Brown has a great, gaping hole where his self-confidence should be.

It was weird, the way that being the chancellor was never enough for Brown. His fixation with taking over the prime minister’s job, as he had been promised, was the acme of emotional immaturity. It was Brown’s behaviour over this matter that led to the whispering campaign about his “psychological flaws”. This was supposed to be scurrilous. God knows why. It’s a so-called smear that could have been laid to rest with a simple: “Who hasn’t?” The fact is that a man who craves the validation of being prime minister that much is a man who does indeed have psychological flaws.

Brown’s tragedy is that those flaws were not so terrible. He struggled on with them, bottled them up, and let them grow into frustration, impatience, sometimes rage. Brown’s reserved, Presbyterian upbringing; the trauma of receiving a debilitating injury as a schoolboy; the loss of his political mentor, John Smith, just when they were getting somewhere – all these are things that made Brown who he is. Which is a man so unsure of his own virtues that he can’t stop trying to convince himself and everyone else that he’s one of the good guys. Which anyone with half a brain can see perfectly clearly that he is anyway.

• Deborah Orr is a Guardian columnist

• This article was amended on 30 October 2017 to remove the assertion that Gordon Brown was interviewed on Radio 4’s Today programme to promote his book. He did not appear on the show.