This is a remarkable a book about an extraordinary man who was Dwight Eisenhower’s “right hand” for foreign policy.

Robert Cutler, known as Bobby, was born into a prominent Boston family, the fifth of five sons whose friends had names like Cabot, Lowell and Lodge. He attended Harvard, fought in the first world war, graduated from Harvard Law School and served as corporation counsel for Maurice Tobin, a Democratic mayor of Boston. Then he became an army general, a senior strategist on Eisenhower’s presidential campaign and America’s first national security adviser.

He was also a banker, a poet, a cross-dresser who loved the female roles in amateur theatrical productions and a closeted gay man at the center of a gay White House love triangle.

Ike’s Mystery Man is the perfect title for Peter Shinkle’s fine biography, from which the big headline is about Cutler’s shepherding of a disastrous 1953 executive order under which the lives of thousands of federal employees were destroyed just because they were gay.

Like nearly all gay men of his time, Shinkle writes, Cutler “struggled profoundly to find, recognize and accept his sexual and romantic orientation”. And like far too many, he was perfectly willing to promote discrimination against other homosexuals, if it would help deflect suspicion from himself.

Shinkle, the great nephew of his subject, learned of Cutler’s sexuality in a casual conversation with his aunt long after Cutler’s death. In his book he has done a superb job of untangling the cross-currents of postwar Washington, when the Wisconsin senator Joseph McCarthy, his evil henchman Roy Cohn and their ally, Vice-President Richard Nixon, made themselves notorious by persecuting former communists and current homosexuals who they argued were mortal threats to the government. (Twenty years later, Cohn became the tutor in the dark arts of bluster and bullying for another Washington disrupter: Donald Trump.) Ironically, Eisenhower’s executive order banning gay people from the federal government and all its contractors was part of his effort to extinguish McCarthy’s work.

Shinkle correctly identifies two landmark events for gay Americans which happened in 1948 and helped launch “a revolution in public perceptions”: the publication of both Gore Vidal’s bestselling novel The City and the Pillar and Sexual Behavior in the Human Male by Alfred Kinsey, which reported that 37% of all males had had at least one homosexual experience in their adult lives. Shinkle has an intriguing theory: “By raising homosexuality in the public consciousness, the Kinsey and Vidal books may have contributed to a period of heightened discrimination” after the relatively permissive years of the second world war.

“A strange climate of paranoia and dishonesty permeated Washington,” Shinkle writes of a city “where vicious hunts for homosexuals were led by men themselves suspected of being gay [like McCarthy and Cohn], where people laughed as [Joseph] Welch and McCarthy sparred maliciously over the words pixie and fairy … and where senators practiced the art of gay blackmail against political foes …

Homosexuality was simultaneously everywhere but nowhere, suspected but not proved, concealed but then revealed, loathed and labeled a security risk – but then giggled about. Amid it all, from … deep inside the White House, Bobby and Skip were doing their best to serve President Eisenhower and fight Communism.

Skip was Tilghman “Skip” Koons, a gorgeous 27-year-old Russian-speaker who Cutler recruited for the National Security Council staff. How Koons’s homosexuality failed to prevent his employment is a mystery: Shinkle speculates that because of his experience with the CIA and in naval intelligence, he may not have received a formal FBI background check. He was recommended to Cutler by an ex-lover, Steve Benedict, who Cutler knew from the Eisenhower campaign.

Incredibly, at the height of the gay witch-hunt in 1954, Benedict also joined the White House as its security officer – and the FBI director, J Edgar Hoover, invited him to a cocktail party to celebrate his appointment.

Cutler did leave his White House post before the 1956 election, possibly because of fears his orientation might become public. But he returned for Eisenhower’s second term. He was completely besotted with Koons, to whom he wrote wild love letters which Shinkle quotes:

Out of all the concerns in my world this interest in you and yours predominates. As if some electric current, invisible but sure, pulsed across the sea to communicate between us. I hope this can always be so …

Cutler was a workaholic who told Koons he was taking an amphetamine, Dexedrine, to wake up and a barbiturate, Seconal, to sleep. Koons was less than half his age. Eventually, the older man presented the younger with a 163,000-word journal about their relationship, which may or may not have gone beyond hand-holding. Benedict inherited that diary and 600 letters between Cutler and Koons. He gave it all to Shinkle, a research bonanza which made much of his book possible.

All three men lived on a razor’s edge. In 1957, a White House correspondence clerk named George Dame was arrested in the men’s bathroom in the library of George Washington University by a member of the vice squad. Dame named two other gay White House staffers: one of them told the FBI Cutler was gay. Hoover took personal charge of the investigation. Once again, mysteriously, he took no action, possibly because it would have damaged his relationship with the president.

Hoover himself had a “spousal” relationship with his deputy Clyde Tolson which according to his biographer Richard Powers was “so close, so enduring, and so affectionate that it took the place of marriage for both bachelors”. But since Hoover’s was the only Washington house that was surely never bugged by the FBI, what actually happened inside his bedroom is lost to history. Truman Capote called Hoover a “killer fruit … a certain kind of queer who has Freon refrigerating his bloodstream”.

Cutler was a wildly emotional gay man: after his dying mother gave him her mother’s wedding ring, he inscribed it with their initials and his own and wore it for the rest of his life. When he was 67 he boasted to his diary that he was “emotionally immature … And I thank God for it. For the alive freshness of early and penetrating feelings, rushing through me like a wild west wind.”

But there is much more to this story than the private life of a tortured gay man. It covers all the major policies and controversies Cutler interacted with as a White House official, including American-organized coups in Guatemala and Iran, the fight over a security clearance for J Robert Oppenheimer, American nuclear policy in the 1950s, and the response to the launch of Sputnik, the first manmade satellite, by the Soviet Union.

It also provides a revealing portrait of the power of the military industrial complex, against which Eisenhower warned the nation in his farewell address. The bank Cutler worked for in Boston had close ties to the United Fruit Company and the secretary of state, John Foster Dulles, had been managing partner of the New York law firm that represented Standard Oil of New Jersey. Both corporations received considerable protection when the justice department tried to pursue them for anti-trust violations.

This is a constantly surprising work by a former investigative reporter. His sophistication, the multiple secrets he uncovered and the breadth of his narrative make it one of the most rewarding books of popular history I have ever read.

• Charles Kaiser is the author of The Gay Metropolis: the Landmark History of Gay Life in America, a new edition of which will be published in 2019