A few years ago, I went to a retirement event for someone who, in his late 80s at the time, had spent more than 60 years as a professor at New York University. He had been embedded in every aspect of academic life, from mentoring and research to fundraising. Over the years he had managed to teach 100,000 students the university’s Introduction to Psychology course. Ted is one of those institutional pillars who can tell you what the place was like in 1965. These days, most people don’t last more than four years in one job.

I walked into Ted’s party thinking it would be full of students and teachers, but I was wrong. There were guests from his theatre days, people who hung out at his favourite piano bar, as well as the “techno music” people. Ted had a smorgasbord of identities he had fostered in the city, some related to his career as a scholar, but most not.

Now an emeritus, Ted still comes to his office for a few hours each day to socialise. Some might think that he’s having a hard time “letting go” – he’s so married to his job, he doesn’t know what to do with himself. But Ted’s not suffering from separation anxiety. In fact, quite the opposite.

Throughout those 60 years, he maintained a healthy separation from his work. Being a professor was never his be-all and end-all, his central identity, pulling the strings on his self worth like a puppet master. Instead, it was one of many facets of his life. And although he loved parts of his job, there were plenty of aspects of it he didn’t. Ted had as many career lows as he did highs, yet he was always able to maintain a healthy relationship with being a professor.

I like to contrast Ted’s approach to what the modern careerist often tries to achieve; the one who values “purpose-driven” professions above all else, and who is willing to make huge sacrifices in order to succeed in them.

Gone are the days in which jobs were just jobs. Despite all the talk of work-life balance, the importance of having psychological distance from our careers is not an especially fashionable value. Even those who want that distance – who don’t define themselves by their professional achievements and lean on other identities instead – are often afraid to admit it. They worry that people will interpret their mindset as a sign that they’re apathetic, lack passion, aren’t dedicated enough.

And it’s not surprising that we’ve got here, given the messages we’re chronically exposed to. Social media feeds are full of carefully curated posts that make “doing what you love” seem easy and natural, often inducing painful comparisons with our own situations. Business leaders talk about how finding passion at work is what has enabled them to achieve success. This narrative has led us to believe that if we only play our cards right, we can enjoy the same kind of unqualified contentment.

Having researched the sources of work-related unhappiness, and studied thousands of people at various places along the spectrum of fulfilment, I worry that this narrative – that loving your job is a necessary condition for both happiness and efficacy – is not only problematic, it’s dangerous for our mental health.

There are three main reasons: first, many of us will never find this love, yet there’s a “pluralistic ignorance” that makes us believe everyone else has. Pluralistic ignorance is when you think everyone around you is engaging in some behaviour, or has a set of beliefs, when it isn’t actually true. (My favourite example of this in social science is from a paper called Pluralistic Ignorance and Hooking Up. It’s about how, on college campuses, there’s a belief that everyone is having one-night stands and loving them – but actually no one really does.) Believing that other people love their job more than you can breed jealousy, resentment and, in some cases, depression. These negative experiences lead to poorer performance and, sometimes, selfish behaviours such as hiding knowledge from co-workers. The quest for love can bring out the worst in us.

Second, we’ve come to believe that falling in love with your career should happen instantly – within months of starting the job. But in reality it takes time and experience to appreciate the good things. It can be hard to take in the highs without experiencing some lows, for example, including long stretches of doing nothing but monotonous, tedious work that characterises a large part of many roles. Ted weathered these storms by dedicating himself to other pursuits, but the modern careerist interprets such stretches as red flags that say “this isn’t right for me”. Ironically, expecting to love your job means you’re more likely to quit it.

Third, even those who do find love are often at risk of negative outcomes such as chronic stress and burnout. Love means dedicating your whole self to the job, and, in turn, feeling every failure and setback like a punch to the gut. Having multiple “contingencies of self-worth” – things you can turn to feel good about yourself when one aspect is going poorly – is the key to buffering yourself from stress.

Love can distort your perception of reality, too: loving your job could also mean ignoring clear signs that it isn’t, in reality, a good fit for you. It might encourage you to put up with the toxic boss and unfair workload, rather than looking for something better elsewhere.

In his book The All-or-Nothing Marriage, relationship scholar Eli Finkel describes how we’ve gradually come to expect our romantic partners to meet all of our needs: personal growth and self-discovery, romance and friendship. We’re seeing the same trend with our careers, wanting them to provide us with an all‑encompassing sense of meaning. The consequences of this mindset are similar to what we’ve seen with marriages: there are a lot more divorces.

To develop a healthy, long-term relationship with our careers, we should kiss the “love” trope goodbye. Instead, let’s try to have a healthy amount of psychological distance from work. Relish parts of your job, but don’t expect to feel passionate about the whole thing. And, above all, drop the expectation that to be really good at something, you’ve got to be head over heels.

• Tessa West is a professor of psychology at New York University.

Further reading

The All-or-Nothing Marriage: How the Best Marriages Work by Eli J Finkel (Dutton, £16.99)

The Squiggly Career: Ditch the Ladder, Discover Opportunity, Design Your Career by Helen Tupper and Sarah Ellis (Portfolio Penguin, £16.99)

The Happiness Trap: Stop Struggling, Start Living by Russ Harris (Robinson, £12.99)