“Everybody’s got the right / To their dreams,” goes the lyric in Stephen Sondheim’s musical Assassins, sung by a chorus line made up of men and women who have tried, with or without success, to kill the president of the United States.

The infamous group includes John Wilkes Booth, Charles J Guiteau and John Hinckley Jr. An updated version might add Thomas Crooks and Ryan Routh for allegedly trying to assassinate Donald Trump last year. But there is no place for Richard Pavlick, the would-be killer of John F Kennedy you probably never heard of.

After more than six decades of obscurity, Pavlick’s story is finally told in depth in The JFK Conspiracy: The Secret Plot to Kill Kennedy – and Why It Failed, written by Brad Meltzer and Josh Mensch. It details how, three years before Lee Harvey Oswald struck in Dallas, a retired postal worker with a warped sense of patriotism and car full of dynamite tried to turn himself into a suicide bomber.

“This is the first full book written about Richard Pavlick and the incident and that was pretty surprising to me,” Meltzer says via Zoom. “But I think the truth is that when Kennedy finally dies this story just can’t measure up to that story so it becomes that footnote. Then you realise, wait a minute, we want to examine this person, we should examine this life.”

Pavlick was born in New Hampshire in 1887 and served briefly in the army, including in the first world war, but spent most of his career as a postal worker in Boston. Paranoid and deluded, he felt that America was under threat from foreign influence. In 1955, the book relates, he tries to form a Protestant war veterans organisation that excluded Catholics and Jews.

Meltzer, whose books include The Lightning Rod and The Escape Artist, elaborates: “He clearly has mental health issues. He creates clubs to make sure that Jews can’t serve in the military, if you’re not like him, you can’t serve in the military. He’s the kind of guy who, if your music’s too loud, comes and threatens you with a gun and says: ‘Lower it.’

“He’s not all put together and by the way, of the four people that have killed presidents, all of them have that instability built in. That’s why you do take a shot at the leader of the free world. But the one thing I can say is he’s very alone in the world and he’s very angry and that’s a potent combination.”

Pavlick retired to the small town of Belmont, New Hampshire. The JFK Conspiracy describes him as a man full of grievances – “a chronic complainer”, as one townsperson put it. From his days in Massachusetts, he felt a profound antipathy towards the Kennedy family. In his view, Joe Kennedy Sr was trying to buy or steal the 1960 presidential election for his Catholic son.

Meltzer and Mensch put these attitudes in a wider context. The Ku Klux Klan hated Catholics as well as Jews and African Americans. Evangelist Billy Graham covertly and overtly tried to thwart Kennedy’s candidacy. The Reverend Norman Vincent Peale, author of The Power of Positive Thinking (later a major influence on Trump), declared: “Faced with the election of a Catholic, our culture is at stake.”

Pavlick, then 73, duly voted for Richard Nixon in the bitterly divisive election, writing in a letter to an acquaintance that, if the Republican did not win, he would “put a hex on Kennedy and his family millions”. Kennedy emerged with a narrow victory. Soon after, Pavlick drove to the Kennedy family compound in Hyannis Port, Massachusetts, and saw the president-elect greeting a crowd at an airport in Cape Cod.

This was not mere sightseeing. Over three weeks Pavlick bought eight sticks of dynamite. On 1 December 1960, he packed some clothes and other belongings in his 1950 Buick and prepared to leave Belmont for good. He drove to Washington and on to Palm Beach, Florida, where he booked into a motel.

By now he was a desperate man. Pavlick wrote to an acquaintance in New Hampshire: “Am here at the end of my rope… [if] I die someplace in Florida you will read about it perhaps within 3 weeks.” In another postcard he wrote: “I am never going to N.H. again… I came down to die you will see it on T.V.”

Pavlick knew that Kennedy would be in Palm Beach, enjoying the Florida climate before taking on the duties of commander-in-chief. On the morning of 11 December, Pavlick parked outside Kennedy’s compound and waited for the president-elect to leave his house to go to mass at St Edward Catholic Church, a prominent institution that had served Catholic residents since the 1920s.

The authors write: “Hidden in the trunk, tucked under blankets and mixed with assorted junk and tools, are seven sticks of dynamite. Affixed to that dynamite is a wire that runs from the trunk into the body of the vehicle, toward a small trigger mechanism.

“All the man has to do is activate this small trigger to blow up the vehicle – and everything around it. According to one expert, if detonated, the amount of dynamite is powerful enough to ‘blow up a mountain.’ And right now, the man’s hand is moving toward the ignition.”

Pavlick planned to slam his Buick into Kennedy’s black sedan, triggering a massive explosion and killing them both. But when the president-elect emerged, he was not alone. Pavlick could see he was accompanied by his wife, Jackie, and children Caroline and John Jr, who was less than a month old. He hesitated.

Although he was ready to kill Kennedy, he baulked at wiping out his young family, so he resigned himself to trying again another day. His chance was lost. Four days later he was arrested by police working off a tip from the Secret Service, who had been alerted by a postmaster back in Belmont who was unnerved by postcards sent by Pavlick – the postmarks indicated that he was stalking Kennedy across the country.

Taken to the county jail, Pavlick, told reporters: “I had the crazy idea I wanted to stop Kennedy from being president. Kennedy money bought the White House and the presidency.”

Had he gone through with it and succeeded, Kennedy would never have delivered his “ask not what your country can do for you – ask what you can do for your country” inaugural address, dealt with the Cuban missile crisis, or declared his ambition to put a man on the moon by the end of the decade.

The presidency itself would also have been different. Meltzer reflects: “JFK was our first celebrity president. We’ve had many famous presidents: there are plenty of presidents waving from the backs of trains, from Abraham Lincoln and before, where crowds come. But JFK and Jackie are selling something very different.

“They’re selling a presidency of celebrity and fame and wealth and image and we’ve been chasing that presidency ever since. For some people it’s Reagan, for some people it’s Obama, for others it’s Trump, but all of them have been cosplaying the JFK presidency. It’s a hollow pursuit because Camelot never existed and that shiny, glossy surface on the outside often obscures reality.”

He adds: “For me, though, the one thing at the core of it that isn’t hollow, is hope. And what the Kennedys gave us was, they unleashed a level of hope that was incredible. That’s what JFK and Jackie did better than anyone.”

But Pavlick, who was committed to a mental institution and died in 1975, was quickly forgotten. At the same time he was arrested in Palm Beach, two airliners collided midair above New York City, killing 44 people. Newspapers at the time obsessively covered this plane crash and its dramatic images effectively knocked Pavlick off the front pages.

Kennedy went on to assume the presidency in January 1961, apparently unshaken by the incident and without a major security shake-up. Meltzer continues: “We’ll never know what’s in his head but, as far as we can tell, he didn’t seem to care.

“He wasn’t bothered and whether that sense of invincibility came because he survived WW2 and was like, ‘The Japanese and the Nazis couldn’t kill me so some guy with dynamite is certainly not going to’, or he thinks that way just because he’s a handsome guy and he’s good at getting what he wants, or he thinks that way just because he doesn’t know all the details, we don’t know.

“But it makes what happens in November of 1963 almost a bit more heartbreaking because you feel like they kind of had to already deal with it once. The head of the Secret Service at the time says it’s the closest call they’ve ever had in Secret Service history to a sitting president.”

The JFK Conspiracy also chronicles Kennedy’s wartimes heroics and notorious extramarital affairs as well as Jackie Kennedy’s grace, resilience and astute creation of the Camelot myth. Kennedy’s luck ran out in November 1963 when, riding in an open-topped Lincoln Continental, he was assassinated by Oswald in Dallas.

Conspiracy theories still swirl that Oswald did not act alone, turbocharged by director Oliver Stone’s 1991 film JFK and the QAnon movement. Trump promised during his reelection campaign that he would declassify all of the remaining government records surrounding the assassination if he returned to office. Meltzer, however, is not holding his breath.

He comments: “I do not think there are magical papers that are being hidden that are going to show us things that no one knows about. If you want to know who killed JFK, I’ll tell you right now. If you look at the 60s, right after JFK is shot, we said JFK was killed by our enemies at the height of the cold war: the Russians did it, the Cubans did it.

“If you look in the 70s, right after Watergate happens and distrust of the government hits new heights, who killed JFK? It was our own government did it, it was an inside job, it was the CIA, it was LBJ. Then in the 80s The Godfather movies peak. Who killed JFK? It was the mob. If you want to know who killed JFK, it’s whoever America is most afraid of at that moment in time decade by decade.”

As a septuagenarian, Pavlick was unusually old compared to the four men who did assassinate presidents, all of whom were in their 20s or 30s. For many people it is hard to accept that one person can wreak such havoc and end the life of the most powerful person in the world.

Meltzer reflects: “Whether it’s that Lee Harvey Oswald shot or the John Wilkes Booth one, the reason why conspiracies exist is because they allow us to believe in a reality that makes us feel safer. The idea that the whole world can come undone by one person is such a scary thought. It’s so much safer to believe that if you wanted to take down a president, you need a cabal.

“You’d need tons of people, you need billions of dollars, you need a plan and chalkboards and string and it would take all these things to make it happen. That’s such a safer thought than the idea that one person on one day can just do it himself. Over and over we refuse to realise that’s how it’s always going to be. It just takes one person to change the world.”

• The JFK Conspiracy is out now