On my last visit to the Azores – those black volcanic islands set adrift in the middle of the Atlantic – the underwater photographer Andrew Sutton and I watched a pod of sperm whales at play. Six or seven animals – it was sometimes difficult to say how many there were – spent hours rolling around one another in what biologists call a “socially active group”. Their ebony black or dove grey bodies were in contact throughout. Fins stroked flanks, gentle jaws bit one another. At one point, two of the whales brought their pugnacious, square foreheads together in what looked like an act of philosophical communion. We were not only watching the world’s biggest predators, but also the animals with the largest brains, communicating with one another in the most intense, sensual manner.

Hal Whitehead and Luke Rendell have been studying sperm whales, and other cetaceans, for a combined total of 50 years; Whitehead’s last book, Sperm Whales: Social Evolution in the Ocean, has been a bible for my own investigations into these enigmatic animals. Zoned-out freaks of the 1960s generation saw these marine mammals as aliens; now it’s time to ask serious questions. Do cetaceans have a culture? Can a non-human culture exist? And if so, how does that change our responsibility towards other species?

Modern whale science is a recent discipline. It only really began with that psychic shift in the late 60s. Until then, whales had been a locus of fearful wonder, or an industrial resource. That changed when the young scientists Roger Payne and Scott McVay lowered their hydrophones into the ocean and recorded the song of the humpback whale. In an instant, an animal hitherto regarded as dumb, and unable to protest its abuse, acquired a voice capable of delivering a melodious, fluid, abstract threnody. It was a 2001 moment. We realised our error, and our insignificance. Payne’s recording, Songs of the Humpback Whale, made the album charts in 1970. As it established its culture, the whale invaded ours.

Whitehead’s and Rendell’s chapter on whale song is key to their provocative, brilliant book. We now know that these songs are evidence of what they term the first non-human cultural revolution. Humpbacks on the east coast of Australia, for instance, learn their song from those on the west coast. The songs change from year to year, subtly shifting in tone and composition, as if a new hit version had been released and every whale were keen to follow the fashion. The songs are clearly part of the mating process, but why do they display such an elaborate expression (some last for more than 24 hours)? The authors examine two theories. First, that the songs signify belonging and bettering among the males, a reassurance of culture in common, a secret code or call to assembly.

Second, females are attracted to new songs, excited by their novelty, the booming and trilling. Indeed, at a recent event at the National Maritime Museum, The Whale: An Exploration, the BBC sound recordist and composer Chris Watson installed a state-of-the-art sound system to demonstrate whale song as it would be heard in the ocean. With the drum’n’bass notes still reverberating in our collective bones, he revealed that these sounds may act as foreplay between male and female whales, physically bringing the latter into oestrus.

What is the evidence for cultural transmission in other cetaceans? Socially learned foraging techniques include the use of tools – dolphins will employ sponges to protect their beaks from the poisonous barbs of fish as they search in the sea’s sandy bed. For orca, or killer whales, diet itself is a cultural statement. Fish-eating or meat-eating orca define themselves by their consumption, to the extent that captive whales accustomed to dining on seals will starve rather than eat proffered salmon. Bottlenose dolphins, enduring our intimate observations, enslaved in dolphinaria, yield precious clues to their sense of self-identity. Not only do they use signature whistles for themselves, but they will mimic others’ whistles in an act of complex communication. Plaintively, dolphins in captivity can remember the whistles of companions from whom they have been separated for 20 years or more.

But it is to the sperm whale that the authors (and this reviewer) are continually drawn. Where the humpback has its song, the sperm whale has its clicks, produced by that giant head – in fact, an extended nose. Filled with spermaceti oil (early hunters believed to be the animal’s semen, hence its name), it functions as the natural world’s most complex sound generator. The oil has bioacoustical properties, amplifying the clicks produced by an organ behind the animal’s nostrils. Jet-engine-level decibels stun its prey – large fish or squid. But the sounds can also be used more subtly, to communicate.

Together with fellow scientists Ricardo Antunes and Shane Gero, Whitehead and Rendell have discovered five clans of Pacific sperm whales, defined by the sequences of clicks they use to communicate, like a South African Xhosa-speaker. Loosely assembled in vast groups of thousands, these tribes interact with one another yet remain discrete, self-identified by their click dialects. It is a kind of ethnicity, a sign of “the way we do things”. In this limitless water world, a whale’s home is other whales. Bound by communality, sperm-whale culture expresses a collective individuality: “We” and “us” may be more important than “I” and “me”. If that isn’t a lesson for their Homo sapiens cousins, I don’t know what is.

The final chapters of this groundbreaking and beautifully produced book pose stunning questions, and tease out outrageous answers. If culture exists in cetaceans, have they developed an equivalent moral sense of right and wrong? Yes, say the authors. Whales and dolphins observe rituals of the dead and exhibit grief. Could they, then, express spiritual sentiment, founded on values and belief – even a sense of religion? Perhaps. All this only underlines a pressing need to address the notion of non-human rights for such animals. (One of my Twitter correspondents recently proposed a whale United Nations.) We don’t yet understand human culture; how can we hope to understand cultures predicated on the alien, unfathomable world of the ocean? Maybe those 60s freaks had a point after all. If Whitehead and Rendell had a fraction of Nasa’s budget, we might work out our complex relationship with the only “extraterrestrial” life that we know to actually exist.

They write with wit and good humour as they take on their critics. If cetacean culture exists, why don’t whales fight to defend it? Just another good characteristic of the whale. The authors admit that “cetaceans don’t make hard axes, or anything else that endures”, but nor do they start wars. Their brains have been evolving for far longer than ours. Those sperm whales I watched off the Azores were there before the islands erupted from the sea. And if Herman Melville’s predictions of a flooded world in Moby-Dick come to pass, they may yet outlive us all, and “spout their frothed defiance to the skies”.

• Philip Hoare is the author of The Sea Inside (4th Estate). The Cultural Lives of Whales and Dolphins is available from Chicago.