Now here's a funny thing – a history of the music hall by a former Conservative prime minister. Former PMs and party leaders have tended to write books about highly respectable subjects (and themselves, of course). But John Major – an unusual politician and unusual PM – has written, in honour of his dear old dad, an uncommon book about one of the most extraordinary, vital and creative chapters in British theatrical history, another influential contribution to world culture improbably emanating from this tiny island.

He has his work cut out. Music hall is a notoriously elusive subject. It emerged from the semi-legitimate underbelly of the 18th-century London entertainment scene, from upper-class dining clubs and middle-class taverns and working-class so-called penny gaffs. This is what Major strikingly calls "the pre-natal life of music hall". There was always, from the beginning, a sense of transgression, the saying of the unsayable, the thinking of the unthinkable. Sometimes this simply meant hair-raising levels of ribaldry, sometimes stark truths about city life, sometimes mad flights of verbal and visual fantasy. Sometimes it was directly political, such as the mock trials, with "barristers" and "juries". Eventually, with licensees dancing around the vagaries of the authorities, dedicated establishments sprang up – the original music halls, which were often little more than that, simple venues where popular music could be heard by a hard-drinking clientele.

The alcoholic and the artistic were always closely linked: some performers were paid a bonus of a penny for every pint that was drunk, and many of them were simply paid in kind with booze, which swiftly curtailed their lives. Once the commercial possibilities of the halls were established, by the mid-1850s, smart entrepreneurs such as Charles Morton built increasingly splendid theatres, offering excellent food and fine wines in safe and sumptuous surroundings. They also sought to tap into the respectable, especially that holy grail of impresarios, the family audience. In time this would inhibit the free and raucous expression of the early days, but that element was never entirely expunged. At its glorious high noon, from the 1870s through to the beginning of the first world war, it was a unique mix of social comment, sexual innuendo, musical brilliance, physical poetry and visual spectacle. It celebrated the grotesque, the defiant, the carnival; above all it was about personality and the give-and-take between stage and stalls. It was frequented by all classes, chronicled by great writers, admired by creative geniuses. Stravinsky wrote a piece inspired by Little Tich, whom Nijinsky idolised; Debussy told the young Chaplin, "M Chaplin, vous êtes un artiste." From 1914, the decline was steady, brought on by a combination of ruthless central control by a handful of managers, the development of new forms, such as jazz, above all by the encroachment of the new media of radio, film and, delivering the final coup de grâce, television.

All this has been told before, and told well, by Dion Clayton, by Mander and Mitchenson, by John Fisher (in the wonderful Funny Way to Be a Hero) and by Colin MacInnes in his superb elegy Sweet Saturday Night. But Major has a motive in telling the story again. It is, he says, a final encore for his mother and father, Tom and Gwen Major; Tom was 64 when John was born, so his career stretches back, if not to the heyday of the halls, at least to a time when that heyday was a living memory. His account of the music hall is tinged with deep filial affection for the lives of performers and the conditions in which they worked.

As might be expected, he is especially acute on the political and social dimensions of the phenomenon: he notes, perhaps approvingly, that audiences in the 1870s often supported the Conservatives, "who under Disraeli had cut the hours of work, while Gladstone's Liberals had cut the hours of drink". And he quotes that anomaly, the comedian-philosopher George Robey: "The man who gets to the top [in England] is almost treated like a usurper. The man who gets kicked to the ground is regarded as almost a saint … what we are witnessing is the deification of inefficiency." He has a fine chapter about the 1894 attempt by the National Vigilance Association (under the superbly named Mrs Ormiston Chant) to persuade the London County Council to oust prostitutes from the music halls by closing down the Promenade in the Empire Leicester Square; the audience rebelled. The Promenade, where people could move around freely, was the last vestige of the loose and raffish origins of the music hall.

Major has an interesting aside about the presence of rent boys among the audience: even in the centre of London, the music hall was only a step away from the demi-monde. Of course it was: the theatre is always about display, about bodies, about flesh and blood. Major spares a kind word for Mrs Chant, but his sympathy is with the performers.

As it always is in this rich, generous book. He never moralises, but always celebrates. The art of evoking dead performers is a very tricky one, and sometimes it eludes him. Nothing can bring the kilted Scot Harry Lauder, the most famous and (richest) of all music hall artists, back from the dead; and Major rather throws in the towel with Vesta Tilley ("she was, quite simply, supreme"). But elsewhere he sends in the clowns quite brilliantly, parading them before us in all their extraordinariness. Boys pretending to be girls, men pretending to be women, women pretending to be men, white men pretending to be black men – a sort of reverse universe: Little Tich, 4ft 6in tall, a curiosity from birth, having an extra finger on each hand, double-jointed, pigeon-toed, overweight, and who stopped growing at the age of 10, described by Lucien Guitry as the world's greatest genius; the young Dan Leno, billed as "Little George, Infant Wonder Contortionist and Posturer" – no wonder Dickens loved him (he said of the Infant Phenomenon that he would "make headway"); Albert Chevalier, author of "My Old Dutch", the prince of costers, christened Albert Onesime Britannicus Gwathveoyd Louis Chevalier, known as the Kipling of the Halls.

Major is especially strong on the women. He writes movingly of Jenny Hill (1848–96): "She was more than a great performer – she was a feminist trail-blazer, a professional success, at a time when women could neither vote nor borrow money without a male guarantor; when the professions were closed to them and their only employment choices were low-paid factory or shop-work, or domestic service. Jenny Hill not only earned success on her own merits, she remained in control of her career: there were no Svengalis for her. She did it her way, and took the knocks." He quotes one of her many searing lyrics, which suggest a performer on the brink of the Blues: "I've been a good woman to you / And the neighbours all know that it's true / You go to the pub / You 'blue' the kids' grub / But I've been a good woman to you." This is a discovery: I had never heard of her, but Vesta Tilley described her as "the greatest artist we ever had on the variety stage". Major is wonderfully warm-hearted about Marie Lloyd and her audience: "She made mistakes in her life – as they did. She drank – as many of them did. She loved unwisely – as many of them did. Her language could be basic – as theirs was. She lived their lives vividly and in public – and they loved her for it."

In judgments, or non-judgments, such as these, Major shows himself to be compassionate and in touch, as the phrase goes, with real life. Uncommon politician indeed. He is fascinating about the cross-dresser Anne Hindle, who married, under the name Charlie Hindle, her dresser, Sarah; and tellingly quotes Ella Shields's great number, "Burlington Bertie", a sublime example of popular poetry: "I'm all airs and graces, correct easy paces / Without food for so long, I've forgot where my face is." Shields, Major notes with satisfaction, "died in 1952, aged 73, after collapsing on stage from a heart attack in front of three thousand people at a holiday camp in Margate", one of the few happy and not untimely deaths in the annals of music hall.

Major nobly honours his parents and they life they lived in these pages. The affection is palpable and anything but pious. "Whatever the deprivations … it was a world full of life, with each performance a kaleidoscope of colour and contradiction: the beautiful and the bizarre; the glamorous and the grotesque; the romantic and the raffish; the comedic and the crude." In short, he gets it. I hope that many people who may not have been naturally drawn to the subject are seduced into its charms by the author's name.

They will be additionally rewarded with some wry autobiographical asides from this most self-aware of politicians: "The flops, the let-downs, the days without work, the lash of critical opinion," he says of his patents. "It was not until years later, with the political critics poised, invective flowing and the national audience restive, that I fully understood all the emotions that had been so familiar to them." As he says of Marie Lloyd, she "learned the perennial truth, that once you are a target, any dart will do". John Major writes whereof he knows.

• Simon Callow's Charles Dickens and the Great Theatre of the World is published by Harper Press.