Joe Swift & Margaret Drabble

Joe "I didn't do that much gardening as a child. I wanted to play football and my mum gardened around us. I remember once growing an apple tree from a seed and it got to 6in tall, then mum got landscapers in and my tree disappeared. I remember going mental with everyone when I got back from school, but Mum wasn't very sympathetic.

"In those days, the 1970s, everybody's garden was unbelievably traditional. There was a lawn, cottage-style plants and traditional climbers round the outside. I remember thinking it was a bit twee and dull, but looking back, ours was very nicely planted.

"When I began gardening, I started to take over. Being a professional and still living at home, I was a little bit more gung ho. I thought of it as a lab, pruning things a bit drastically, and Mum would come home and say, 'Oh my God!'

"It's ironic that I subsequently went into the media and now write about gardening, but the difference is that it's mainly practical, rather than creative fiction. When I first started writing, I suppose I did feel a little self-conscious, but no pressure. Mum's always said that with writing you have to start and finish something and get on with it, rather than dream or talk about it, and I suppose that's what I did when I was given the opportunity.

"A couple of years ago, I changed her garden. She's got incredibly good taste, but without a designer's eye. The new garden has lots of hard surfaces, quite a big pond to bring in wildlife, and a log stack for insects and beetles. There's a lawn with a generous path running around it. The planting is loose and naturalistic, and works a lot with her existing shrubs. Every time I go round there, I get excited to see how it's changed."

Margaret "Joe was always an outdoor boy, but I wouldn't say he was very useful in the garden. He filled it with rabbits, until finally they died.

"I've learned a lot from him. He would say, 'You don't want geraniums, they're granny style' (I still have geraniums). I have a weakness for red and know I have to ask permission for that. He did teach me that I should think in more architectural terms: big upstanding plants and tree-shaped things. Joe is very keen on euphorbias and I've got them at our house on the coast.

"I was very impressed when he embarked on being a gardening journalist. He was never very keen on essays in school, but obviously enjoyed writing about something he was interested in.

"The saddest thing of all: for my 50th birthday, he gave me a maple that we planted in our Hampstead garden, and when we sold the house I made a special provision to go around and get it. But once we'd moved, I somehow didn't think it was right to go back."

Andy & Marion Sturgeon

Andy "My parents had the typical hideous garden. A lawn; a bed of hybrid tea roses that were all diseased; a bed of dahlias – they went through this ritual of digging them up and storing them in the garage; and these apple trees that my brothers and I were enlisted to pick and prune. I always associated that job with being cold and damp.

"We spent an awful lot of time outdoors, so the garden was a big part of my life, but I never considered myself a gardener. The whole thing was a bit of a chore.

"I had this small patch that was mainly a pet cemetery, but I did grow some seeds. I was probably six or seven. I remember growing phlox and nasturtiums.

"I suppose I learned the art of gardening from my mum – going through the borders, cutting back, adding compost, that sort of thing. She is very industrious, methodical, and I think I got that from her.

"Initially, I rebelled against everything they had: the amount of work that was needed; the borders that were quite bleak in winter. I went for much less intense gardening, and not having such a lot of bare soil. I could never get my head around the fact that if you had bare soil, you had to fork it over to make it look nice. My idea is to plant and leave it. I also didn't grow dahlias or roses, but for the past 10 years or so I've come round to them.

"Mum still has the rectangular lawn, but now with more fashionable shrubs such as pittosporum, trachelospermum, bamboos… There's a lot less bare soil now."

Marion "I wouldn't let Andy loose designing my garden for me, because my garden is mine, just as his designs are his. There are certainly colours I won't have – I don't do red, yellow or orange if I can avoid them. Pastels are my thing. I think Andy's taste is somewhat similar.

"I used to grow dahlias, and that was a standing joke because he mentioned it in an article and I protested. I like herbaceous plants and a background of attractive shrubs, so there's something there in winter.

"I've had to have a garden that's basically a playing field with plants round the edge – for dogs, small children and football. I will occasionally say to Andy, 'Come and look at my so and so' but we don't make a point of it. He comes here with three energetic boys and his dog, and I have my dog. The most important thing is to have space for the dogs and children."

Sarah & Faith Raven

Sarah "I remember more the atmosphere than specific memories — being in the garden and hanging out with both my parents while they were doing things, walking through, smelling the flowers. I was very keen on wildflowers from a young age – that very much formed the basis of my love of gardening. Wild collecting dictated what we had in the gardens. On holidays in the Mediterranean, my parents collected plants such as artemisia and tender bulbs – we had them long before they were fashionable. Euphorbias, too.

"I learned from my mum an absolute passion for plants and how they go together. And very much from her is the idea of bringing the odd sprig of this or that into the house, bringing the outside in. She loves scent and that I definitely inherited from her. I remember a rhododendron called 'Polar Bear' with a wonderful scent which we always had in our sitting room.

"Differences? She thinks I care too much about presentation. She doesn't like fuss and bother."

Faith "I have two gardens. One is a rhododendron garden in the West Highlands, and there's the garden where Sarah grew up in south Cambridgeshire, where there are mixed borders and self-seeders. At that time I think she was more interested in ponies, but she did also have a love of growing things. I remember her doing bunches of wildflowers for the home. She's got these very nice little vases where she always has flowers – irises, say."

"We've got a lot of pathways going around our borders, and I have fond memories of Sarah and her twin wandering round the paths in matching dresses. They might even have been holding hands.

"Sarah came to the gardening world as a florist, and being a florist she is much more organised than I am. She would have to keep the garden carefully weeded for her cut flowers and plant things in rows, whereas I'm more interested in plants sowing themselves and going where they want to grow. Sarah's also interested in very bright, vibrant colours. I tend to have a strong colour as a spot plant, with other, less bright colours around it."

• For open days at Perch Hill, Sarah Raven's garden, go to sarahraven.com. Faith Raven's two gardens are open to the public; docwrasmanorgarden.co.uk and ardtornish.co.uk for details.