As Vincent Fantauzzo tells it, there were a “few forks in the road” that could have led him to become a criminal not an artist. There was the time he was arrested for assaulting two men outside a KFC, retribution for an attack on his little brother. His stint growing and dealing marijuana, during which he filled every inch of a rented four-bedroom home with his hydroponic setup. Or the night he stole a family friend’s car and crashed it into a street sign.

All of this may come as a surprise if you only know Fantauzzo as one of Australia’s most successful portrait artists, a four-time winner of the Archibald prize’s people’s choice category who has painted everyone from the former prime minister Julia Gillard to actor Heath Ledger. It’s a line of work that has introduced him to high-flying friends like Baz Luhrmann, Matt Moran and Richard Roxburgh, as well as his wife, Asher Keddie – whom he painted a portrait of in 2012 and married two years later.

Now, for the first time, his new memoir, Unveiled, lays bare the painful details of his early life: growing up in public housing in Melbourne, contending with poverty and an abusive then mostly absent father. He was sexually abused as a child by an older boy for a year, left school at age 16 after being told by a teacher that he was destined “to fail in life” and, for a time, looked poised to enter the world of organised crime. He never learned to read or write, owing to the dyslexia that went undiagnosed until adulthood. And while Fantauzzo has talked about his dyslexia before, he’s never publicly discussed his childhood. So what made him ready to talk?

“I did an Australian Story [in 2019] and I got this great response, but I felt like I only told half the story … the dyslexia,” he says. In conversation, Fantauzzo is open and generous in his responses about the difficult topics – though he often has to ask me to repeat my questions. (“See, this is what it’s like talking to a dyslexic person!” he says with a laugh.)

He knew that another famous friend, the late Michael Gudinski, had wanted to tell his own story – but died before he got the chance. After the music industry executive’s death in 2021, Fantauzzo came to the realisation that he needed to “tell the whole story”, which he thought might help those who’d been dealt a similar hand in life.

It felt like the right time, but that didn’t mean the process was easy.

“It was extremely difficult – like, really, really difficult to actually not sweep the past under the rug any more, and face things that happened and things that I’d never spoken about to anyone, even Asher,” he says.

It was only once work on the memoir was under way that Fantauzzo revealed some of the tougher details of his childhood to his wife, including that he’d been sexually abused. His dyslexia meant Fantauzzo could not pen the book himself, but rather worked with a ghost writer, Penguin’s Craig Henderson. Every day they would speak on the phone, an emotional process full of both tears and laughter. Later, Henderson would read him back chapters as he went, or Fantauzzo would use a text-to-voice program to review the work-in-progress.

“As I was revealing it through discussions [with Henderson], I would slowly reveal things to Asher as well,” he says. Having talked about it once with his ghost writer made it easier to discuss a second time with his wife – and gave her a new insight into her husband: “It helped her understand my reactions to things generally … [like if] there was something about children on the news, she’d wonder why that impacted me so much. Everything started to explain itself.”

There are still more difficult conversations to be had. Fantauzzo’s father is dead, but when we speak he still hasn’t talked to his sibling or mother about the content of the book. “I imagine in the next few days I will,” he says with a nervous laugh.

But while Keddie lent a hand editing the book – she would review chapters and suggest amendments, or flatly tell her husband to remove certain details – she was also “a little sceptical and nervous” about Fantauzzo writing it at all.

“Asher is a very, very private person,” Fantauzzo says. “And as well as exposing a lot about me, this is also a lot about us and the family as well.” That includes their difficult journey with IVF to conceive their son, Valentino, a much yearned-for brother to Fantauzzo’s son Luca from a previous relationship.

But by and large, Unveiled is about Fantauzzo’s early life and then his incredible – if unlikely – ascent in the art world. He started drawing in school where, unable to follow the lessons, he would instead sketch portraits of those around him. A portfolio earned him a place in a visual arts degree at RMIT, his entry into a world where many of his peers came from privileged backgrounds – an experience that gave him “quite a big chip on his shoulder”.

“When I went to university, I’d never even been to an art gallery. I didn’t even know who Picasso was, I had no idea about anything in the art world at all and I was completely intimidated,” he remembers. “I just felt like a total outsider.”

His big break came with his now famous portrait of his friend Heath Ledger, completed one day before the actor’s death in 2008. The painting went on to win the People’s Choice award at the Archibald – and kickstarted his career, albeit at tragic cost. Fantauzzo gave the portrait to Ledger’s mother, who in turn donated it to the Art Gallery of NSW, wanting the piece to have a wider audience. It hasn’t been displayed for years, something that irritates both Fantauzzo and Ledger’s father, Kim. “Kim has written to them [to say] if you don’t want it, then give it to someone who does,” Fantauzzo says. “A lot of people want somewhere to go to look at him and pay their respects. Here’s the chip on my shoulder coming out – we donated the painting to the art gallery. It’s for the people.”

But despite becoming one of its darlings, Fantauzzo still feels like an outsider in the art world. He wants to change a lot about it: he hates that galleries take a 50% or 60% cut on sales; he wants to make the idea of becoming an artist “popular and accessible” to young people; and he is keen to shift attitudes around “selling out” – he and Keddie are brand ambassadors for Alfa Romeo.

“I think there’s an [idea] in the art world where you have to be a bit gritty, ride a single-speed push bike and live in a sharehouse. And when you stop doing that, you’ve sold out. But it just means that you got paid. When I became an ambassador for a car company and people were like, ‘What does he think he’s doing?’ … I’m just as ambitious as everyone else that wants a great career.”

Growing up the way he did, Fantauzzo says he will now gladly “take the option to fly business class” where he can, naysayers be damned. And he knows how differently things could have gone.

“I guess I have a pretty good resilience to [things]. I saw some friends that didn’t … and they’re no longer here, or they’re in jail. I do feel very lucky that somehow, I landed on my feet,” he says.

“Some people, they just don’t have a choice but for it to destroy them,” he adds. “I don’t know what that is. It’s why I do portraits, because I want to figure out how and why some people can get through things but still have empathy and vulnerability … And I do wonder, if it’s just a matter of finding that one thing [you love], like art – that saved me.”

• Unveiled by Vincent Fantauzzo is out now (Penguin, $36.99)