One of the Royal Ballet’s finest hours last year was the anniversary revival of Wayne McGregor’s Chroma, in which five of the company’s home cast were joined by five dancers from the Alvin Ailey company. The chemistry that sparked between those 10 performers, grounded in such different dance traditions, with such contrasting styles of attack, expression and musicality, was exhilarating in itself – but it was made even more so by witnessing a genuine moment of diversity on the Royal Opera House stage.

The Royal Ballet’s forthcoming season doesn’t feature anything quite so physically – or politically – stirring, but it does build interestingly on the idea of opening up the stage to other companies. As a tribute to the late Kenneth MacMillan, the Royal is not only programming a revival of the choreographer’s popular ballet Manon, but also presenting a mini-season of his shorter works, which will be performed by Northern Ballet, Scottish Ballet, English National Ballet and Birmingham Royal Ballet and the company’s own dancers.

This is a much too rare instance of the British ballet scene uniting for a single project (rather than quarrelling over status and funding), and it sets an excellent precedent. I can’t, however, admire all of the repertory choices. However many times I sit through The Judas Tree, the ballet’s unfocused violence and misogyny remains an anathema, and I’m easily irritated by the relentless strut of Elite Syncopations (although it will be fun to see if a competitive spirit arises from the casting).

But it has been too long since London saw Concerto – with its lovely, limpid central pas de deux – and longer still since it saw the rarely performed La Baisée de la Fée. MacMillan’s setting of the Stravinsky score is not without flaws – the music itself is problematic – but the ballet is a fascinating encapsulation of his choreographic psyche, weaving elements of darkness and trauma into the fairytale libretto.

Early in his career, MacMillan positioned himself as a rebel against ballet’s reliance on fairy stories and fantasy; but the big fairytale classics such as Sleeping Beauty and Nutcracker have remained cornerstones of the Royal’s repertory, and this season includes the long-awaited premiere of the company’s new Swan Lake.

Much is riding on the success of this production, given the issues that dogged its predecessor. While Anthony Dowell’s choices as director were mostly admired, the cluttered, sometimes luridly gothic designs by Yolanda Sonnabend were much disliked, for their overbearing weight on the ballet’s delicate poetry. This time around, the sets and costumes will be by John Macfarlane, who has a good track record in balancing visual boldness with spatial discretion.

More controversial is the choice of Liam Scarlett as the overall director of this production. The 30-year-old choreographer is unquestionably talented, possessed of a musical and inventive facility for making steps, and an instinct for dramatic imagery. But he has not yet proved himself as a creator of story ballets, and works like Frankenstein and Sweet Violets have shown him stumbling over focus and pace. To entrust so pivotal a new production to a choreographer still learning his craft is a risky proposition.

Personally, I think the Royal are justified in taking the punt. Scarlett will be working to a fixed libretto and score, and his role as a choreographer will be restricted to tweaks he makes to the existing Petipa-Ivanov material. If the Royal has a duty of care to its classics, it also has a commitment to nurturing in-house talent, and this commission comes at a timely moment in Scarlett’s career. Immersing himself in the deep structural logic of a major classic, the choreographer may come out with a far more confident grasp of how to solve the narrative problems of his own ballets.

Prior to Swan Lake, Scarlett gets another choreographic outing in the season, with the revival of his 2014 ballet Age of Anxiety. This comes as part of another anniversary programme that marks the 100th anniversary of Leonard Bernstein, the composer who played a major role in the development of American classical music and ballet.

The Royal’s all-Bernstein triple bill will also be a showcase for the company’s trio of associate choreographers, with Scarlett’s ballet accompanied by new works from Christopher Wheeldon and Wayne McGregor creating new works to Bernstein scores. No details have been given on their choice of music, though McGregor has already named his collaborating artist as ceramicist and author Edmund de Waal, who wrote the family memoir The Hare With Amber Eyes.

Earlier in the season, there is more new choreography, in a programme that boasts the startling mixed-bill pairing of Arthur Pita and Twyla Tharp. Tharp’s last creation for the Royal was the exuberant but unwieldy story ballet Mr Worldly Wise, made in 1995; her latest work returns to the more predictable terrain of pure dance. It’s a reworked and expanded version of her 1973 ballet As Time Goes By, which will now use the whole of its original Haydn score and include an extended duet, inspired by the partnership of Steven McRae and Sarah Lamb. We rarely get to see Tharp’s choreography in Britain, and even if this isn’t an entirely new ballet, it is always a pleasure to see her fierce, logical choreographic brain at work.

It also promises to make a bracing contrast with Pita’s ballet – his first for the Royal – which promises to figure elements of the deranged, dark romanticism that is signature mode. It’s inspired by the 1928 silent movie classic The Wind, starring Lillian Gish as a young woman exiled to the prairies, and sent slowly mad by the rawness of the landscape and the sexually predatory men among whom she’s been stranded.

With rebuilding work still going on at the Linbury Studio, there are none of the smaller-scale experimental projects and guest companies that typically add an edge of unpredictability to the Opera House’s dance programme. But still the 2017-18 season features a good tranche of new work; and with the Tharp commission following on from this year’s Crystal Pite, we can dare to hope that the Royal will never again consider it acceptable to programme season after season of exclusively choreography by men.

As for the rest, it’s a balance of contemporary box office favourites (Wheeldon’s Alice and Winter’s Tale): 19th-century classics (Giselle and The Nutcracker) and a welcome revival of Frederick Ashton’s Sylvia. Despite a libretto that – even against the low standards of ballet – comes riddled with outrageous nonsense, this 1952 setting of Delibes’s entrancingly coloured score contains some of Ashton’s finest choreographic invention. Its titular heroine also ranks as one of the wittier, more challenging roles of the ballerina repertory – a wonderful vehicle for dancers such as Francesca Hayward, Yasmine Naghdi or Natalia Osipova.

The romantic lead, unfortunately, is one of the sappier of the male principal roles, but the men in the Royal will be getting a different kind of bonus next season with the appointment of Carlos Acosta as company repetiteur. Even while he was still dancing, Acosta was unusually generous in the informal advice and encouragement he gave to his younger colleagues; it’s excellent news for the company that he now has an official coaching position from which to hand on the inspiration, experience and expertise he’s acquired from his long career.

• This article was amended on 5 April – the Royal Ballet are not staging a production of Romeo and Juliet in their 2017-18 season as previously stated.