"As long as I've been writing music, I've dreamed of getting rid of the performers," the Arkansas-born composer Conlon Nancarrow once said. A communist jazz-trumpeter who fought against Franco in the Spanish civil war, he found many things about his home country lacking: in particular players who could keep time with his extraordinary polyrhythmic compositions. The result was that Nancarrow ended up writing solely for the automated player piano, creating rolls of demanding music resonant with barrelhouse, Bach, Dixieland, Art Tatum, blues and Stravinsky. His music was at once determinedly American and yet like nothing else before it.
On July 4, American Independence Day, you can hear some of Nancarrow's work played by the pianist Joanna McGregor in the unlikely setting of Richard Rogers's Lloyd's of London. McGregor will also be playing pieces by the equally inventive and demanding composers John Cage and Morton Feldman. Where, famously, Cage was able to "play" silence, Feldman's last string quartet, encouraging varying degrees of improvisation from Nancarrow's unloved players, might last for more than five hours. In music, Feldman liked to say, scale, not form was the challenge.
How will these daunting pieces of mid-century modern music sound in the Jules Verne-meets-Blade-Runner, sci-fi vastness of Lloyd's? You must be the judges, because this year the City of London festival brings 20th-century music together with buildings by masters of their art. Since many of the spaces chosen - Lloyd's, the glazed top floor of Norman Foster's 30 St Mary Axe (the Gherkin), James Gibbs's Great Hall at St Bart's Hospital - were neither intended, nor shaped, for the performance of live music, many of the festival's concerts promise to be experiments in experimental sound.
There has always been a close relationship between music and architecture, experimental or otherwise, in terms of structure, pattern and aesthetics, even though sound ultimately describes immaterial space. Plainchant, for example, somehow belongs to Romanesque abbeys, even though its origins are much older, just as Bach is all but synonymous with baroque churches. For better or worse, Wagner conjures images of the fairy-tale, alpine fantasmagoria of Neuschwanstein, the Sleeping Beauty castle built by Wagner's indulgent patron, Ludwig II.
The avant-garde music of the 20th century has its architectural counterparts, too: John Cage fits well with the superbly crafted, philosophically underpinned minimalism of Mies van der Rohe; Daniel Libeskind's Jewish Museum in Berlin could be Schoenberg's apocalyptic Moses und Aron set to architecture, including, at the centre of his zig-zag building, the void at the end of Schoenberg's daunting work.
Iannis Xenakis, the brilliant composer, engineer and architect who worked for Le Corbusier between 1947 and 1960, produced what he called "polytopes", spaces where colour, light, sound and structure worked together in new ways to generate sensual and spiritual uplift. The pilgrimage chapel of Ronchamp and the church and chapel of the monastery of La Tourette, where Xenakis's mark is unmissable, are some of the most numinous spaces shaped in recent times.
Yet, just as Bach transcends domes and Cage cannot be contained by the geometric insistence of van der Rohe, Xenakis's music sought to escape the very architecture it had helped to create. He began to experiment with overlapping compositions, with sounds projected from multiple sources. He wrote music that could be performed on several floors of a building simultaneously, and he wrote the intense, wave-like music that, fused with an unsettling light show, inaugurated Renzo Piano and Richard Rogers's Pompidou Centre in 1977.
In 1984, Luigi Nono performed his new opera Il Prometeo during the Venice Biennale in an ark-like laminated-timber structure designed in collaboration with Piano. The ark stood inside the abandoned darkness of the church of San Lorenzo. The performers moved up, down, around and through this architectural music box allowing the sound to explore the space while constantly diverting the expectations of those listening.
As part of the City of London festival, Psappha, the music-theatre ensemble formed by Tim Williams in Manchester in 1991, performs works by early avant-garde composers Berg, Webern and Schoenberg in Wren's church, St Andrew's Holborn. Well, Wrennish, anyway. The body of this, Wren's largest parish church, was blown away to the accompaniment of the unheavenly music of the Blitz; it was replaced by a more or less faithful reinterpretation of the original by the architects Seely and Paget, completed in 1961. Berg's Piano Sonata is an 11-minute piece from 1908 that, although much influenced by Schoenberg, Berg's tutor, flows in a way that Schoenberg's music refuses to do. Just as there are no sudden junctions, breaches or visual full stops in Berg's sonata, so there are none in the reconstructed St Andrew's.
Equally, the restrained, gentlemanly English baroque of St Andrew's may well be a suitable consort for Webern's clear, cool Four Pieces for Violin and Piano (1909-10). Like St Andrew's, Webern was a victim of war. Denounced as a "cultural Bolshevist" by the Nazis, his career was curtailed in the 1930s and 1940s, and in September 1945, at the time of the Allied occupation of Austria, he was accidentally shot and killed by a US soldier.
The folly of violence is just one of the many themes in Schoenberg's disturbing Pierrot Lunaire, a demanding half-hour mix of performance, verse and music that will have Wren, Seely and Paget turning in their graves.
Maxwell Davies's Eight Songs for a Mad King, first performed in London in 1969, a dark, slightly demonic blend of acting, music and verse (by the Australian poet and novelist, Randolph Stow), will be played in the gloriously apposite setting of the Great Hall at Bart's Hospital, Smithfield. This largely secret yet palatial 18th-century room, designed by James Gibbs, is reached by a flight of stairs flanked by ambitious canvases executed for free to prove the superiority of English art by William Hogarth.
The "mad king" in Davies's piece is George III, a monarch who tried to teach his bullfinches to sing in English (or German?), who loved architecture, and whose spirit can be glimpsed in the great works Gibbs carried out at St Bart's. Berio's Folk Songs (1964), although some are dark and unsettling, might come as relief after Davies's portrait of regal madness.
Debussy's Preludes under the all-but-floating dome of Wren's St Stephen Walbrook; Messiaen's Quartet for the End of Time in the perpetual gloaming of St Bart's; and those pioneering Americans in Lloyd's - the music programme of the City of London festival offers a musical landscape as rich and as resonant as the millennium of architecture daring to contain it.
· The City of London festival runs from June 27 until July 13. Box office: 0845 120 7502.