I was born in Mile End, brought home from hospital to a house on the Old Ford Road, and about these first city years I remember only this: the layout of a corridor; bedroom curtains drawn against the evening’s light; the way butterflies flocked to buddleias growing on derelict land. Our presence was circumstantial: neither of my parents was from London. My mother had grown up in Gravesend, my father in South Africa and then Surrey, my grandparents having been from other places again. We were, I suppose, the sort of people that suburbs are made for, except that we had neither the money nor the inclination.

Just after my fifth birthday we left the city for a small town on the edge of north Dartmoor. I started a new school and lost the accent I had borrowed from my childminder. I learned about weather, which in cities happens between doorways and bus stops, shops and tube station entrances. When it rained, as it often did, coming in sheets or as a fine, penetrating mist that soaked you as soon as you left home, the water from the bath taps ran brown with peat. At the edges of the roads deep gutters became temporary streams, and up on the hills the ponies, huddled in the lee of the tors, stood with their heads low and their hindquarters facing the wind. And this remains my most enduring image of home: coming down off the moor in the rain towards a town lit up against an early dark, the drift of chimney smoke and the promise of shelter.

For the next seven years spring brought the smell of gorse, which is like coconut, and summer came with bracken. I swam in rivers where their progress was arrested into pools, scratching my legs on the stones around them. In autumn there were bonfires. In winter, rain again – and always, beyond the playing fields, at the ends of streets, through windows, the moor, as solid and as absolute as faith. I wanted this to be mine, but despite my enthusiastic learning of the names of hills I was only ever perched there.

When we left again, returning in the mid-90s to an East End that hovered on the edge of gentrification, it didn’t feel like any sort of coming home. I had forgotten what I knew of the city and smiled at strangers in the street. I felt hemmed in. Concrete ended in concrete and it was impossible to see out of it, even if you could get high enough, which is virtually impossible on the reclaimed marsh that stretches from Tower Bridge to the Thames Barrier.

I was 12, and what I saw as the gross unfairness of our move became the focus for all my early-adolescent angst. In response I tried, through an act of imagination, to reclaim the place that had never quite been mine to start with. I wrote about hills and I drew them, again and again, in pencil on the backs of envelopes or in pen in the margins of exercise books. I read about them. I learned everything I could as though knowledge were the same as occupation, and I tried, by force, to root myself into a place I had already left. This, I think, is what made me a writer: this attempt at transposition and the belief which underpinned it, that we might write ourselves into existence. We might tell ourselves home.

• Sight by Jessie Greengrass is longlisted for the 2018 Women’s prize for fiction and is published by John Murray.