There is an almost life-sized cut-out of Barack Obama in Stan Greenberg's conference room in his office beside Washington's main railway station, a short walk from the dome of the Capitol. Even in an office full of political memorabilia, collected by Greenberg during a life on the campaign trail, it is striking in its scale, towering over the election posters and signed photographs from world leaders on the wall.

Bill Clinton, Nelson Mandela and Tony Blair also figure in Greenberg's trophy room. They smile for the cameras with their arms hooked around him, the scrawled messages confirming that Greenberg - regarded as one of America's pre-eminent pollsters and political strategists since he helped Clinton first get elected in 1992 - played a key role in their rise to the top.

Greenberg may have crystallised the big ideas that got Clinton and then Blair elected - "I decided about the formative issues that brought them both to power," he says - but in all his experience of politics, he admits he has not quite seen a political leader like Obama.

"It is [his] energy and engagement," Greenberg says. "This was a very different election, a total change in the way people contributed and participated - the way all of civil society kind of swung to the Democratic side. Obama was masterful in tapping it - but was also a product of it."

The other aspect that sets Obama apart from the leaders Greenberg has worked with (he was on the sidelines in last November's election, polling for other Democratic candidates and support groups) is the scale of the economic crisis confronting the new president. The crisis appears in danger of overshadowing even Obama's revolutionising approach to politics, but Greenberg is not so sure. "The change in the nature of the crisis has given the political project elevated meaning," he says. "What I have realised with Obama is that the project becomes bigger when the crisis is bigger."

That is striking for Greenberg. In his career as a pollster and adviser, he has been drawn to candidates who promised change - Clinton, Blair, Barak, Mandela - helping to craft the central ideas of their campaign, and using sophisticated polling techniques to analyse if those ideas were being well received by the public.

"[My candidates] were all change agents. They all had change elections. They were all self-conscious that they were throwing out incumbents, that the country was going to hell. People voted for them thinking they were voting for change agents; sometimes they were meant to be transformative. These were big things. They were not small elections."

Clinton and Blair set out to modernise their parties and their countries. Mandela wanted to move South Africa beyond apartheid towards reconciliation. Barak wanted to loosen the hold Orthodox Jewish parties have on Israeli politics, and start peace negotiations with Syria and the Palestinians. But in each case, their supporters' hopes of seeing real change were thwarted. Is that Obama's trajectory, too?

"Every great leader disappoints," says Greenberg. Because, as he makes clear in his new book, Dispatches from the War Room, after every election comes frustration and betrayal, large and small, public and very personal.

Recalling his former partners in the political struggle, Greenberg says: "They were all strong; they all found the right moment to articulate their project - but all of them struggled to succeed [after being elected]. In every place that we have operated, there are great institutional forces that make it difficult to bring change.

"In every one of these elections, as soon as the election was over, the finance minister, or the treasury secretary, or the chancellor, had their meeting and said, 'This is all well and good, these promises you made in the election, but now let's face the realities.' [The leaders] were immediately struggling to meet their promises."

Obama, however, has the advantage of lowered expectations. He comes to power at a moment of historic crisis; Americans accept that there are limits to what the current president can do. "I think he has much more time," says Greenberg. "I think people recognise it will be years before he will be able to move things. The worry is that after they give him a little space, the insider forces will take over, things will grind to a halt and he won't be successful."

This has been Greenberg's experience throughout his years at the frontline of politics, both in America and abroad. Clinton could not get his economic plans through Congress and was overly focused on deficit reduction, much to Greenberg's disappointment. Blair waited too long to try to reform schools and the NHS, and then hitched his future to that of George Bush after the 9/11 terror attacks.

Greenberg has always been wary of advising politicians on foreign policy and national security, but he was so disappointed with Blair's support for the Iraq war, and for Bush's policies on the Israeli-Palestinian dispute, that he found himself screaming at the television. "The thing I was most uncomfortable with was finding myself cheering for the vote against the Iraq resolution in the Commons. I am never in the position where I am rooting against my own team, but I found myself doing that.

"You have to remember there was a larger project: health care, education. The overall project for transforming the party and public services was still ongoing. These were my friends and comrades in arms. They were uncomfortable - including Cherie - with what he was doing [in Iraq]. I wasn't the only one who was uncomfortable, but I never considered resigning."

The disillusion with Blair deepened, Greenberg writes, with his refusal to follow Greenberg's advice that he acknowledge his mistakes on Iraq. "I don't think he wanted to entertain that idea. I know it - I had those conversations. He wouldn't go there," Greenberg says. "So I could understand wanting to make changes [in the campaign team] if you wanted to move with this new project."

The break with Blair, which came during the 2005 elections, was painful nevertheless. Greenberg was sidelined for Mark Penn, another Washington pollster and his arch-rival, who had supported the war in Iraq. "It felt like I had been punched in the stomach," Greenberg writes. "[It] was a slow motion marginalisation over six months. There was never a specific moment ... I never had a conversation with Tony Blair except after the fact. They thought they fired me. It just didn't happen."

Unlike Penn, Greenberg seems less concerned with the demographic calculations of elections than much grander causes, such as the notion of a "shared project". Unlike other recent offerings from Democratic strategists, notably Penn, there is no talk of the soccer mums or Nascar [white, working class] dads of previous elections. (According to Greenberg, Penn, now widely blamed for Hillary Clinton's defeat, was "biased, self-deluding and overly optimistic" in his polling for Labour in the 2005 elections.)

Greenberg, in his book, is looking beyond that, at how to build a more egalitarian society, and how to restore trust in political leadership. That sensibility makes him a bit of an oddity in the cable and internet age. A small man with a mild manner and somewhat scholarly way of speech, he does not fit the pop-culture image of a political operative: outsize personalities, barking out soundbites and lovingly tearing apart their rivals on cable television.

Which is not to suggest that Greenberg is an outsider in Washington nowadays. In the new post-election reality of the capital, Greenberg and his wife, the Connecticut congresswoman Rosa DeLauro, are entirely in the loop. Rahm Emanuel, who as White House chief of staff probably ranks as the second most powerful man in America, has been a friend since the Clinton days: the two share a ritual 6am conference call with other veterans of that campaign. Greenberg and De Lauro were also invited to the official lunch on the day of Obama's inauguration, and to his first social gathering at the White House, to watch the Super Bowl. A broadly grinning DeLauro was on the front page of newspapers with the Democratic leadership in Congress the day after Obama's titanic economic recovery plan was passed.

Greenberg is candid about that proximity to power in the book. After the 1992 election, he had a weekly 15-minute meeting slot with Clinton at the White House. The breakdown in their relationship came after the Democrats were defeated in the 1994 congressional elections. "The meetings after the election were tense but it never crossed my mind that I wouldn't be there to plan the comeback," Greenberg recalls. "Clinton and I had weathered many ups and downs. I presumed I was too central to our shared political project." Yet while Greenberg continued to conduct polls, the meetings stopped. "Even the Secret Service looked at me differently."

And yet, if not for his fellow pollsters, Greenberg still betrays a sneaking affection for the leaders he once served, despite the cavalier personal treatment and broader betrayal of what he thought was a shared cause. "Maybe they all have to disappoint," he says. "But they try."

• Dispatches from the War Room: In the Trenches with Five Extraordinary Leaders, by Stanley Greenberg, is published by Thomas Dunne at £18.99.