Alan Bennett never expected to be writing in his 90s, with a novella published next week and a film in post-production. There was a time when he didn’t expect to reach 70. In 1997, a cancerous growth was found in his colon that had already begun to spread. “The surgeon didn’t think I’d got a chance, really,” recalls Bennett, 27 years on. “So yes, it is a slight surprise that I’m 90.”

He attributes these bonus decades to two younger men: his partner, the magazine journalist Rupert Thomas, 58, and the director Nicholas Hytner, 68, with whom he has worked on 11 theatre and screen projects. “It was luck that I met Rupert – over a shared taste in paintings, really – and also that I met Nick, more or less at the same time, around 30 years ago. It’s been the best period of my life. Without Rupert and Nick, I’d be nothing, I think.”

Although Bennett and Hytner had already collaborated on The Madness of George III, which premiered at the National Theatre in 1991, Bennett dates his creative renaissance to The History Boys (2004), his most commercially and critically successful work, about a group of Leeds sixth formers and their eccentric teacher, Hector. “I’d long given up hope of writing anything really fresh, but it was so enjoyable. The actors treated me as part of the cast. The first preview was my 70th birthday and I couldn’t believe I was still going at that age.”

Twenty birthdays later, he still is. The only concession to the decades is that he apologises for not getting up from the sofa to greet me at his north London home, where Kate, a friend of the couple, is on hand while Rupert is at work.

“This is a recent development,” Bennett explains. “I’ve fallen once and that totally altered Rupert’s approach. He doesn’t like to leave me on my own for long, and so we have people who come in.” Health-wise, he says, “the main problem is I can only walk a short distance. Since I had cancer, I’ve been regularly in hospital one way and another for other things.”

The poet Philip Larkin, about whom Bennett has written, thought about death and illness a lot. “All the time!” Bennett agrees. But that’s not his attitude? “No. I think I was more of a hypochondriac before I had cancer. And then that trumped everything else really. I’m not in the least bit worried about people knowing I’ve had cancer, but you don’t like to flaunt it because of a superstition, in my case, of thinking it will come back. But the ‘good thing’ about cancer is that you have regular checkups, so you can ask straight away about anything. Except I then had an aneurysm and then open heart surgery. Although I was encouraged that, on one consultation, in the waiting room was Michael Palin, who I had always regarded as the picture of health and energy.”

Palin has spoken publicly of his cardiac surgery. For medical professionals who were fans of English comedy, it must have been extraordinary to have on the same clinical list members of both Monty Python and Beyond the Fringe, in which Bennett’s career began, with Peter Cook, Dudley Moore and Jonathan Miller.

Bennett has come to accept that “old age is a subject. You write about what you’re given. And I hadn’t expected to have anything to write about by now.” Killing Time is set in a home for the elderly. He started it a few years ago but set it aside because Allelujah! (2018), his most recent stage play, was set in a hospital elderly care ward: “I didn’t want to become the bard of geriatric medicine!”

On the cover of Killing Time is a scent bottle with a hammer and sickle logo. One care home resident, who worked in Whitehall during the second world war, met the Soviet foreign minister Vyacheslav Molotov at a peace conference and received a bottle of scent made by Madame Molotov, a top Russian perfumier.

Bennett stresses that her career is not a comic flight: “I read about her by accident – in the Guardian, actually. And I thought it was too extraordinary to miss: that’s one of those things a proper writer uses.” (Interviewing Bennett across three decades – and despite his accelerating success during that period – he has never lost this trope of differentiating himself from real authors.) “I remembered Molotov from newsreels. But didn’t know anything about him apart from the Molotov Cocktail.” [A petrol bomb named after a politician who, ironically, was professionally a diplomat.]

As research, he secured a less incendiary Molotov bottle, a scent called Red October: “I couldn’t smell it at all. It got through to Rupert but he hated it! There had been some notion of bringing out a limited edition of Killing Time, impregnated with the perfume. But that plan was abandoned.”

There is a lot in the novella of what might be called sunset sex; a handyman meets male and female residents by arrangement in a garden shed. This extends an increasing emphasis in the work of Bennett’s bonus decades, indicated by the title of the 2011 volume of short fiction, Smut: Two Unseemly Stories, in which middle-class women have striking secret erotic lives. The Laying on of Hands, a 2001 novella, follows an escort worker, while The Shrine and An Ordinary Woman – two new monologues added to the BBC’s pandemic lockdown remakes of the Talking Heads soliloquies from the 80s – also feature transgressive sex within families.

“I’m very forgiving,” the writer admits. “I wouldn’t have anything to write about if I didn’t. It’s partly that I can’t take sexual scandal quite seriously, I suppose. Being gay, and not feeling any need to hide it now, I think it makes you more generous about what people want.”

This year’s Theatre Royal, Bath revival of The History Boys met criticism for the character of Hector (a role created on stage and screen by Richard Griffiths), whose clear pederastic interest in his students goes unrebuked. In interviews over the years, Bennett has always refused to censure Hector, but now goes further.

“Partly because I was so tentative myself about sex, and scared really, I saw Hector as reflecting that. But – we could never really talk about this – Richard Griffiths’s size, in a way, solved the problem, really, because the boys were so unlikely to be interested in him. You couldn’t voice that even at the time, but he wasn’t threatening. Once, when the understudy went on for him, he was threatening. So you realised, in a way, we’d been lucky with Richard.”

But when people now suggest, about other productions, that Bennett is too tolerant of Hector? “My attitude was and is that the boys were so sophisticated that they were more alert to the world than he was. I can’t defend it other than that.”

Similarly, in Killing Time, a male resident, Woodruff, is always exposing himself to women in the home, who either close their eyes or roll them.

“Exactly. The same as the pupils in The History Boys. And I’m sure that is as true sometimes to what happens as the harmful interpretation is. Maybe I’d feel differently if I’d been abused myself, but I never was.”

In Killing Time, the world wars are important reference points for the care home residents. Does that reflect Bennett’s own life? “Yes. The second war – Dad was exempt, as a butcher, from serving. It’s quite funny that butchers were exempt while slaughterers weren’t. Not surprisingly, really, they just carried on by other means. The first war I simply knew from the photo on the piano of my Uncle Clarence, my mother’s brother, who had died in it.”

Bennett wrote a radio feature, Uncle Clarence, about this unknown soldier, and the conflict that claimed him inspired his first screenplay, A Day Out (1972), directed by Stephen Frears, in which a Sunday jaunt for Halifax factory workers in 1911 is clearly a pre‑war idyll for a doomed generation. It is intriguingly book-ended by Bennett’s new film, The Choral, which Hytner is currently completing, in which, in 1916, a Yorkshire choral society, due to perform Elgar’s The Dream of Gerontius, loses its male voices to military conscription and replaces them with younger boys who also sense that this may be their swansong. “I was fascinated by the fact that, if all the men went to war and they were replaced by children, the whole sound of the choir would suddenly change. We just saw a rough cut and what surprised me is that it’s a bit of a tear-jerker, really. Unusually for me.”

When Bennett did national service after school, was there any risk of seeing active service? “Well, there was. In the sense that, after basic training, you were then sent wherever you were going to go. And everyone had told me when I went in that you had to ask to do the Russian translation course in Pontefract, which was officially secret, but you had to keep asking until they took you. I got on it but the rest of my squad was transferred to Korea and some of them were killed.”

Was Bennett a good soldier? “No! Though I wasn’t fazed by having to be neat and kit inspection and so on. I could do all that. And I enjoyed parade-ground drill. I could see why people liked dancing; I could sway along with it.” On the Russian course, intended to provide interpreters or spies for the cold war, Bennett met another future author and playwright, Michael Frayn, who put his schooling to use with Chekhov translations – “I always say Michael drew the cheque on Chekhov,” says Bennett. The lessons may, more tangentially, have seeded Bennett’s works about the Cambridge double agents: the stage play The Old Country (1977) and the TV film An Englishman Abroad (1983).

“It was like university before university,” Bennett remembers. “I went to Cambridge, as did Michael, and the Russian course was like a more carefree version of it.”

Does Bennett still remember his Russian? “No! I used to be able to speak bits because Rupert liked hearing it. But it’s gone.”

Was he ever tapped up to be a spy? “No! I don’t know if Michael was. He’d have been much better material than me. There might have been people in our year who were. Probably the ones we didn’t like.”

Bennett and Rupert still regularly go to their house in Yorkshire, the most frequent setting for his work, including Killing Time, in which the language, as often in Bennett’s work, is stamped with the vocabulary of his home county: “chumping” for chopping wood, “clap cold” for freezing. “The one that I get credited with by [author and lexicographer] Ferdinand Mount is ‘splother’. Anything overdone or too much fuss, my Dad would say: ‘It’s a lot of splother!’ Mount says he’d never come across that one before my work.”

Does he tend to hear dialogue in a Yorkshire accent? “Yes. I often don’t know whether an expression is Yorkshire or standard English,” he says. “I wish I’d paid more attention to my dad. He’d say things like, if you asked him where something was, ‘I’ll ascertain!’ And that, more than anything, is northern. A slightly piss-taking, archaic way of talking. The person I think got that best was Victoria Wood. She caught that idiom in a way no one else did.”

Apart from you? “No. She was better than me. And made me realise I should have listened harder. I went to her memorial service. And they must have made it ticket only, because there were very few people there. Whereas if they’d had open house, as it were, it would have been packed out because she was wonderful and could fill the Albert Hall.”

The big stylistic shift in Bennett’s extra writing time has been the move from drama to fiction. His first novella, The Clothes They Stood Up In, was written and published either side of cancer surgery and chemotherapy. Killing Time is his eighth prose fiction. So why did he start writing for page rather than stage and screen?

“I think it was finding that whereas, if you do something on the stage, there’s all this scene-setting, in prose you can do it in a sentence.” The Clothes They Stood Up In opens: “The Ransomes had been burgled”. “The title led it to write itself.”

The Uncommon Reader (2006), in which Queen Elizabeth II becomes a voracious consumer of literature, is a favourite of many Bennett fans. “Yes. I enjoyed writing that more than anything else ever. Once I had the idea, I didn’t have to do much. I knew how the Queen spoke. It was such a gift.”

The late Queen was also a character in A Question of Attribution (1988), in which Prunella Scales as “HMQ” elegantly interrogated her surveyor of pictures, recently revealed as a Soviet double agent, Sir Anthony Blunt. The durable impact of these works comes from her being a huge historical figure, yet also somehow unknown.

“Yes. She came to Leeds once and Dad and I went out to watch her. As she came by, Dad suddenly took off his hat, and I found this immensely moving. I don’t know where it came from. Any kind of formal behaviour was ‘splother’. But it made me realise he was a loyal subject, as I suppose I was. She was so easy to write about. You had to just touch her and she became dramatic.”

Did Bennett meet her? “I shook hands in a line once. That’s all it was. I was once asked to lunch and didn’t go. I apologised. Because I would have been so shy. And I’m not sorry, because it would have got in the way of anything I wanted to write about her. Nick Hytner met her once or twice and hearing his accounts was just as good from my point of view.”

Yet despite this reverence, he declined a CBE and a knighthood? “Yes. I have a complicated relation to this. Last night, I watched a documentary about Maggie Smith and they showed her being given the CH.” (Companion of Honour, the highest prize of the British state.) “And I … I was very upset when she died, and suddenly it all came back and I cried, which I don’t normally do. The whole relationship between the state and people is very difficult. When I was offered things a couple of times, there was always something that had just happened – usually a war. And you just felt you were being used as laundering somehow. And that’s probably a conceited way of looking at it, but anyway.”

Bennett’s Yorkshire near-contemporary, the artist David Hockney (the two, to their bemusement, are often confused due to late-age thick hair and spectacles) now holds both a CH and also the establishment silver medal, the Order of Merit. Wouldn’t Whitehall say the nation was simply honouring its creative talent?

“Well, I think there’s a sense in which honours are not trinkets, they’re shackles. Even when I took an honorary degree from Leeds University, there’s no formal obligation, but you’re automatically the first person they ask to come to talk. And I’m no good at that. So, after Leeds had asked me once or twice and I said no, I thought I really don’t want to do this. It’s not modesty, it’s laziness.”

The evening before we meet was Maggie Smith Night on BBC Four, including a repeat of A Bed Among the Lentils, the Talking Head monologue that Bennett also directed, in which she played the unhappy alcoholic wife of a vicar, recalling the brief release of an affair.

“She didn’t need any direction. I think the only ‘note’ I gave her was ‘champion!’, my father’s favourite term of praise. There was a bit that completely took me by surprise, even though I’d written it and was directing it. She’s talking about her [secret lover] Mr Ramesh having moved on and her voice breaks but she doesn’t cry. She just edged that emotion in. And I found it so upsetting. She could somehow be funny and sad in the same sentence.”

Smith brought this bittersweet technique to the film A Private Function (1984), inspired by Bennett’s father’s experience as a butcher in the second world war, and The Lady in the Van (stage 1999, film 2015), in which she played Miss Shepherd, a vagrant evangelist who parked her caravan on Bennett’s drive and lived there for 15 years. The character could have been played as grotesque, but Smith somehow made her seem likable and even justified.

“Yes. And also replaced her in my memory. When I think of Miss Shepherd, I think of Maggie, really.” He points across the room. “Stand up and look in the hall. On the wall.”

Above the door to the downstairs lavatory is an English Heritage blue plaque inscribed: “Miss MT Shepherd, The Lady in the Van, Lived Here 1974-1989”.

“She didn’t,” glosses Bennett from the sofa. “It was the other house.” (In one of the shortest ever relocations, they moved from one Camden Town street to the next.) “But Rupert had it made up for my 90th birthday.”

Then, perhaps surprisingly, he reveals of Dame Maggie: “I didn’t know her outside work. I made her nervous and she made me nervous, so we kept our distance.” He agrees, though, that she was one of two natural interpreters of his work, the other being Dame Thora Hird. A story about Hird recording the 1998 Talking Head Waiting for the Telegram, in which an old lady remembers the death of her sweetheart in the first world war, may be the perfect Bennett anecdote – going off in unexpected directions but coming home to a neat autobiographical pay-off: “Thora Hird had in common with John Gielgud – probably the only thing they did share – an ability to access tears at will. In Waiting for the Telegram she cries for a long time and talks through the tears. And, when we’d done it in the studio, nobody said anything for a long time and it was because the camera crew were crying. It made me think of when we were in Beyond the Fringe, we often did cabaret nights and Peter and Dudley could always make the band laugh. And I never did ever. Anyway, after Thora, I thought well, I couldn’t make the band laugh but I’ve made the camera crew cry.”

He politely excuses himself to go for a wee – “my age,” he says, although we have been talking for 90 minutes – and returns with a very Bennettian piece of literary criticism: “Samuel Beckett makes out old age is a kind of drying up, going to dust. In fact, getting old is more liquid than that. You’re always being taken short.”

When his legs permit, he gets to revivals of earlier plays: “I’m glad they want to do them. But I dread it. Normally, when we’ve got something on for the first time, it’s how I wanted it. It’s as near perfection as we can manage. And the second time round, it isn’t.”

He reveals that he hasn’t seen the 2022 movie version of Allelujah!, starring Jennifer Saunders and directed by Richard Eyre rather than Hytner.

“I thought I might get upset by it. I liked my original play. I like Richard Eyre, and I didn’t want to be upset by feeling he’d got it wrong.”

Does Bennett have a sense of his development in his later work? “What late style means for me is being able to dispense with self-consciousness and, as my mother would have said: ‘What will people think of you?’ You don’t care any more. For example, Smut wasn’t particularly liked. And it didn’t matter. You just think: well, at least I’ve got them down on paper.”

As for even later work, “lost” Bennett works occasionally come to light. Denmark Hill, an unmade screenplay, was produced on Radio 4. Hytner found the two extra Talking Heads in a drawer. Might there be more finds from the archive?

“No. There may be stuff lying round but even I don’t think it’s worthwhile. There’ll be bits. I haven’t stopped. If I didn’t write, I wouldn’t know what to do. So I’m always making notes for things.”

Could there be another stage play? “I’d never like to say no. What I’d be flattered by is if the Hallé orchestra could somehow do a concert version of The Choral, with either me or someone else doing the adaptation. It’s a good idea. But I’m no good at pitching things.”

After three hefty volumes of diaries and miscellaneous prose – Writing Home (1994), Untold Stories (2005) and Keeping On Keeping On (2016) – his publishing rhythm suggests that another collection is due around 2025-26.

“There will be. I’d always kept diaries but in a haphazard fashion on scraps of paper. So they resembled a heron’s nest. When I gave them to Faber, they were amazed this kind of performance still went on and it wasn’t electronic in any way. So now we again have the problem of sorting out these scraps and putting them in an accessible form.”

That volume – working title The Far End – will include his diaries since 2016, a signal political period. He was a Corbynista once. “Was I?” He said he “wholly approved” of Jeremy Corbyn. “I did. He seemed more sympathetic than most and was demonised. But really what I want politically is for the Labour party to be in government so that I can stop thinking about politics. I know they can be foolish, but they’re not wrongheaded in the way the Conservatives are. I’ve voted Liberal Democrat sometimes, so I’m not hidebound.”

In July, he was one of the 18,884 residents of the Holborn and St Pancras constituency who voted directly for the prime minister and, despite Keir Starmer’s bumpy first 100-plus days, does not regret it: “There’s always this. Blair had all this fuss about Peter Mandelson when he came in.” The politician resigned from Blair’s first cabinet amid controversy about a loan to buy a house. “So they were bound to go after him for a while. But we’ll see.”

As next Tory leader, Robert Jenrick or Kemi Badenoch or neither? “Well, certainly not Jenrick. James Cleverly seemed more acceptable but he dropped out.”

Donald Trump or Kamala Harris as president? “Oh. Goodness me. You’d have to be insane to want him. But I’m dreading it happening, just the tension of it. I’m afraid he makes me thankful that I’m 90, not young. But, with Rupert being much younger than me, I’m more in touch with younger people than I expected to be, so I do worry for them.”

Killing Time, in common with many Bennett works, features a vicar. Is his interest in religion only literary? “Into my 20s, I was still saying my prayers. Rupert is a militant atheist. In the days when I could walk, we liked going to old churches. But any sign of religious activity – even a children’s corner – the less he liked it.” Does Bennett have any residual feeling for the faith? “I suppose so. I’m a lapsed Anglican.”

Since Beyond the Fringe, he has maintained a side career as a performer and is recording the audiobook of Killing Time, although he is concerned by the character of Zulema, a carer in the home. “Really, I should do her in what I’d like to be a good African accent. But, at the same time, if you do that, it’s prejudiced.” So what will he do? “I don’t know. It’s next week.”

Apart from such self-interpretation, he considers himself retired as an actor. “And I’m thankful. I wouldn’t be able to appear on stage. I’d be terrified of forgetting the lines. It’s bad enough in conversation; I forget things. The last time I was on stage was at a National Theatre gala two or three years ago. Richard Griffiths had died and I played Hector in The History Boys. And I was so nervous. Helen Mirren was backstage and I told her how scared I was of the audience and she said, ‘Fuck ‘em’. I wished I’d been given that advice much earlier.”

• Killing Time is published by Faber and Profile on 7 November. The Choral will be released next year. To support the Guardian and Observer, order your copy at guardianbookshop.com. Delivery charges may apply.