“He was clearly not a performer,” says author and illustrator Michael Kupperman of his father, Joel. “After a certain point, he was not going to become a popular television personality. He didn’t have that kind of ease with the camera. He was very charming as a kid but, as he got older …” He pauses to find the right words. “He’s a very dry fella, you know?”

I find it difficult to imagine my own dad as a media sensation, so it is easy to sympathise with Kupperman’s assessment. But as the most prominent member of the much-loved Quiz Kids radio programme, Joel was, for a time, the most famous child in the US.

Nicknamed “midget Euclid” due to his incredible maths skills, Joel wowed audiences from the age of five, gazumping adults and public figures with feats of mental arithmetic. “Mathemagician” Joel was adored by the listening masses, fawned upon by celebrities and co-opted for photo opportunities by various US statesmen. One 1942 photo caption described Joel as having “black sparkling eyes, a toothless smile, a lisp, and a brain that works like an Ack-Ack gun”.

“They were almost like props,” says Kupperman of the show’s child stars. “There are a lot of pictures of them with performers or politicians or newspaper people, and they’re frequently sitting in their laps – maybe a thing they wouldn’t do these days.”

His father’s fascinating childhood is the subject of Kupperman’s new graphic novel All the Answers, a funny, sad and beautifully illustrated account of his father’s time in the spotlight, and the toll it exacted on his later life. Primarily known for his surreal and hilarious Tales Designed to Thrizzle comics and his richly ironic New Yorker illustrations, Kupperman’s mannered, near-woodcut style is perfectly suited to the old-timey, documentary rendering of Joel’s past. Over 200-odd pages, Kupperman tells the story of the past his father never discussed, a superstardom that saw him netting 10,000 fan letters a week by the time he was seven years old.

“He was a pin-up for a while,” says Kupperman. “He got very strange poems and people writing about him as a potential boyfriend, so it does get into some weird territory. I kind of expect that happens any time someone becomes that famous.”

All the Answers documents the sheer breadth of Joel’s appeal, detailing his meetings with the likes of Orson Welles and Jackie Gleason, as well as mentions of him in the works of luminaries as diverse as Philip Roth, JD Salinger and Nora Ephron. The book begins with an odd, startling admission that Joel once told his son: as a child, he was gifted a dog by iconic comedy duo Abbott and Costello.

“That’s literally the only time he spontaneously shared a bit of showbiz remembrance, ever,” Kupperman says. “Never before or after, so it was a huge shock. I love old showbiz stuff and find it fascinating who he met and spent time with … but he didn’t get anything out of it.”

With this as a starting point, Kupperman originally hoped he could mine his father and extract his memories, fully formed, memoir-style: “I thought maybe the brain was like a massive iceberg. As it melted, dead memories would suddenly emerge, like long-dead caribou, and come floating to the surface.” But to his frustration, he soon discovered that Joel, who now has dementia, had forgotten or repressed almost all of the salient events of the time.

“I legitimately considered hypnotism at one point,” Kupperman says, “but found that repressed memory isn’t generally considered a thing anymore … He’s never come out with anything since then, and all my attempts to get something fell flat.”

In the end, Kupperman’s salvation lay in five huge, flaking scrapbooks kept by his grandmother Sara, Joel’s mother, who was very much the driving force behind her son’s fame. Sara is a fascinating character in the story, and one whose motivations to keep her child in the spotlight seem inscrutable even now.

“It’s not the most sympathetic portrayal in the book, but it’s accurate. Even in the 1940s, when everybody was extraordinarily polite about each other, she was referred to openly in the press as the most stage-mothery of all stage mothers,” says Kupperman. “With our modern knowledge of media and how it works, it’s very easy to say she had no endgame. It’s hard to see what she was going to get out of this, or what was behind it.”

Joel stayed on the programme until his mid-teens, by which time the show had migrated to television, a medium for which he was, as his son attests, squarely unsuited. Once cute, his precocity had now calcified into a smartness of a less charming, adolescent mien. He was considered insufferable by many of his peers. The stress of remaining within the Quiz Kids’ grip, combined with his now being publicly recognisable, began to take its toll. He “graduated” from the show at 16, according to the rules, and eventually left the US to get his doctorate from Cambridge. He returned to a quiet life at the University of Connecticut in 1960, where he worked as a philosophy professor for most of Michael’s childhood.

“The whole thing was very badly handled by his parents and himself,” reflects Kupperman. “He stayed on so long after it was valuable, and on television too, so everyone knew his face.”

One other strain in All the Answers is Kupperman’s quest to interrogate how his father’s singular childhood had affected his own, not least in Joel’s decidedly “hands-off” style of parenting. I wonder whether the extraordinary constraints Joel’s mother placed on her son fed into his becoming a benign, if distant father to Michael.

“That is a little neat,” he cautions. “A lot of parents of that time failed to give their children any guidance, but being on that show from five years old formed his patterns of behaviour way beyond any changes he could make.”

A fair conclusion, he says, a father himself now, is that parenting is difficult. “Everything you’re doing has consequences,” he says. “I think that was a problem with parents of previous generations. They didn’t realise that what they were doing had consequences.”

• All the Answers by Michael Kupperman is published by Simon & Schuster.