“My ambition with this book is to honestly confront the fears some people have about Muslims,” writes Sarfraz Manzoor in his introduction. His hope is to point out the ways in which “they” can become “us”. This was always going to be a challenging, if not impossible, task. “Muslims” in the UK, like “non-Muslims”, behave in ways that are not always clearly attributable to their faith. This is something Manzoor realises and tries to manage by making it clear that there is little doctrinal justification for much of what Muslims claim Islam commands them to do. Once the religious connection is shown to be a myth, other factors fill in the gap to explain why Muslims are the way they are. Manzoor grapples with why they are so convinced that their faith instructs them to do things it doesn’t, and on the way, realises that, well, it’s really quite complicated.

And so the ground constantly shifts under Manzoor’s feet as he tries to distil into neat takeaways what cannot be simplified. The charges he attempts to investigate – 10 in total – include: “They believe homosexuality is a sin”; “They follow a violent religion”; and “They look down on white women”.

Manzoor is to be commended for trying to figure out these things not via cursory research, but through numerous conversations with real people who aren’t chosen to fit his thesis. But these rarely heard voices and experiences (the chapter on teenage boys whose Pakistani parents forced arranged marriage on to them is harrowing) are let down by the form and style of their presentation. There are too many short, disjointed anecdotes and too many characters who are introduced and then abruptly disappear, only to come back again with a reminder that we “last met” this person doing X or Y. The effect is of a deluge of experiences, all rich in detail but with little to pull them together other than a reminder of the premise of the book at the end of every chapter – which is that alongside the bad there is also the good, and the good offers hope. The rhythm is that of a pendulum, constantly swinging back and forth between intense personal experiences of family strife, terrorism, closeted homosexuality and antisemitism within some Muslim communities, and optimism that some within those communities have chosen a different path.

Much of that hope Manzoor draws from the new generation in general, and his children in particular, and how different their lives are from his. He is married to a white woman with “hair the colour of sunshine”, and the most moving passages are in his account of his family’s painful and traumatic rejection of his wife. This experience, of breaking with his family and its ensuing emotional terrorisms, feels formative. It looms large in his life and the book. Manzoor repeatedly returns to the pain of his rupture from his heritage, but also the hope and promise he has found in his wife’s culture, her family’s acceptance of him and the ease with which his children are growing up with a gay uncle. The theme of mixed marriage recurs often, which is more a personal hangover from the less tolerant time, more than a decade ago when the author’s own marriage took place, than a comment about contemporary Britain. Manzoor’s frankness about the insecurities his union raises about betrayal of one’s culture, and his simultaneous seeking of refuge in the marriage, give the impression that his own experience is still so burdened with unresolved conflict that the reader may have been better served by hearing less from the narrator.

Something is being worked out in the background here, as if another book is intruding on this one. The result is that there is at once too much memoir and too little of it to make this an edifying account of how othering occurs. There are some puzzling detours through Manzoor’s poetry, diary entries and stream-of-consciousness footnotes, where his voice becomes distant and removed from the concrete matter of the national crisis he is trying to resolve.

There is, alongside such whimsy and ruminations on identity, a much steelier approach to what constitutes, what should constitute, modern Britain, which is perhaps why Manzoor sounds so distant from his subjects. They are “they” to the author, too. He is judgmental of women who wear the burqa (although once he meets a few, he is chastened). Generally, while doing the valuable service of providing rare insight into Muslim communities from his background, he is still tethered to a much narrower prescription of what they should aspire to be. In this the author is proud to be described as “the living embodiment of the British dream” by David Goodhart, the columnist and author of books about immigration. “I can be angered and appalled by attacks on mosques,” he says, while recognising why a mosque – particularly a large one – might be seen by some non-Muslims as dominating the landscape. “It is reasonable for everyone to have a place of worship but sometimes the size and style of mosques can feel like an admission of cultural insecurity and a resistance to accepting that ‘twenty-first-century Britain is not eighth-century Arabia’.” Muslims, it turns out, aren’t simply misunderstood, they could also make a bit more of an effort to get with the times.

Here is where, again, complexity confounds the book. The “Muslims” of They might as well mean “working-class south Asians who immigrated to the UK in the 60s and 70s in large numbers”. Because They is not a book about what Muslims and non-Muslims get wrong about each other. It is a book about immigration, integration, class and the sort of radicalisation, on both sides of the “they” divide, triggered by alienation and political failure. That is the story of us.

• They: What Muslims and Non-Muslims Get Wrong About Each Other by Sarfraz Manzoor is published by Wildfire (£20). To support the Guardian and Observer order your copy at guardianbookshop.com. Delivery charges may apply.