James Baldwin was always photogenic, but nobody captured him quite like Richard Avedon.

The American fashion and portrait photographer took a stunning shot of the author in 1945, staring innocently into the lens, as if it were a photo booth. And again, he shot Baldwin alongside his mother, Emma Berdis Baldwin, in 1962, donning a suit and skinny tie, smoking a cigarette with an arm perched on his mother’s shoulder, as she laughed with him.

These photos show a far more personal side of the If Beale Street Could Talk writer and social critic than most of us have seen before. They’re on view as part of a group exhibition at the David Zwirner gallery in New York, entitled God Made My Face: A Collective Portrait of James Baldwin.

“I want people to understand his relationship to love, more than anything else,” said Hilton Als, the Pulitzer prize-winning writer who curated the exhibit. “Even though he was erratic, he never lost how valuable people, black or white, were to him. I really want people to take away his ability and his need for love.”

Avedon and Baldwin were high school friends who studied together in the Bronx and worked together at a magazine called Magpie, where Baldwin published some of his earliest essays. They also co-authored a book about American life in 1964, called Nothing Personal.

But strangely enough, with a few exceptions (including Baldwin’s early mentor Beauford Delaney), most of the artists here didn’t know the pioneering writer personally. “The show is about his relationship to artists that he didn’t know and the conversations he never had,” said Als.

They’re shown alongside Glenn Ligon’s portraits of Isaac Hayes, Malcolm X and Frederick Douglass, painted in bold, primary colors. There are also works by the French artist Marlene Dumas, who painted Baldwin and a selection of other notable cultural figures for her series Great Men. Dumas shows a painting of Baldwin alongside portraits of the jazz poet Langston Hughes, actor Gordon Heath, as well as Harlem renaissance writer Owen Dodson and playwright Richard Wright, who told stories of racial violence in the south.

There are also shots of Baldwin taken by his good friend, the Turkish portrait photographer Sedat Pakay. With his camera, he followed the writer around Turkey during the 1960s, which is when Baldwin lived in Istanbul on and off for a decade. It was around the same time he wrote his novel Another Country and was arguably at the height of his recognition, after being on the cover of Time magazine in 1963.

Among the first edition books in the exhibit, there is a copy of Baldwin’s first nonfiction book Notes of a Native Son, from 1955, where he famously wrote: “The American ideal, after all, is that everyone should be as much alike as possible.”

Baldwin’s own pencil drawings are also in the show, including sketches of men that could be interpreted as self-portraits, as well as cartoonish figures. They’re shown alongside a photo by Diane Arbus and Anthony Barboza’s portrait of Michael Jackson.

“I wanted to show Arbus’ work beside Barboza’s photo to show the ways black queerness declares itself or doesn’t declare itself,” said Als. “To see Michael Jackson’s face at 21 is such a startling image, no? Compared to everything Michael did later by bleaching his skin and changing the shape of his head and face.”

“There’s a lot to be learned in this part of the exhibit about black queerness and how it gets mangled in society, in the ways it can be free,” he said.

Baldwin fled to Paris in 1948, when he was 24 years old. His second novel Giovanni’s Room, written in 1956, follows the life of an American man in Paris who has a tumultuous love affair with an Italian bartender. The book – controversial at the time for its homoerotic scenes – was written before the beginning of gay liberation in the 1960s.

“Baldwin said in an interview he had to leave New York in exile to learn how to become himself,” said Als. “I learned that as I grow and admit how difficult it is to be black and queer in New York City, for instance. Now, I can’t imagine what it was like in 1948.”

“Are there any differences between his experience in New York City and mine?” Als asks himself, in monologue. “I don’t know. All I can say is that to make a show like this, you have to put the effort of your own self in there. He is giving so much of himself on the page and in life, you have to respect the effort he made to connect by connecting.”

Baldwin’s accomplishments were not only on the page or in the public eye, but through the lives of friends he touched. He helped inspire Nina Simone to become a civil rights activist and was considered a brother to Maya Angelou.

“Let’s face it, we’re talking about someone who out of extreme poverty whose own parents were slaves, right?” asks Als. “How do you end up in Paris being awarded the Legion of Honour in France? That’s sort of extraordinary.”

“It’s an extraordinary American story about the arrival myth,” he adds. “Not only success but transformation.”

The exhibit doesn’t aim to pay homage to the writer as an icon but rather as a person.

“We can’t underestimate the journey that took him from Harlem to Paris and the journey that took him from becoming a polemicist,” said Als. “It was a lot of responsibility. I really admire him for that.”

• God Made My Face: A Collective Portrait of James Baldwin is showing at the David Zwirner gallery until 16 February