Making the Impossible Possible

What would it be like if a master pianist who died two decades ago could play a concert with today’s leading players? Sound impossible? Well, a new technology developed by Yamaha has done just that.

Yamaha AI Ensemble Technology understands a player’s intentions and responds in harmony with their performance. It works by analyzing a human player’s gestures and sound to anticipate their next notes and then instructs a Yamaha Disklavier™ piano to play accordingly — with a faithful reproduction of the touch of any pianist, living or dead. What’s more, while the Disklavier is playing, a shadow of an imaginary pianist is projected on the stage to give human accompanists a visual cue to synchronize their rhythm and tempo!

Yes, you read that right. This advanced technology provides the experience of playing music with any legend from any time, understanding each other’s musicality and performing perfectly in sync. It goes beyond replicating the touch of a master pianist, allowing live musicians to actually play with his or her avatar.

In a stunning demonstration, a performance was given last year by the renowned Scharoun Ensemble, which was founded by members of the Berlin Philharmonic Orchestra in 1983 and is comprised of some of the greatest string players of the 21th century. For this concert the ensemble was composed of members performing on the violin, viola, cello and contra-bass, and a special Disklavier was configured based on Yamaha’s flagship model CFX concert grand piano and linked to an AI system programmed to replicate the touch of master pianist Sviatoslav Richter, who passed away in 1997. (Interestingly, modern CFX concert grand pianos are a later generation of the Yamaha CF series concert grand pianos that Richter performed on in his later years.)

Ensemble members practiced together with the Disklavier right up to the day of the performance, treating the AI system just as if it were a person, with machine and players deepening their mutual understanding to steadily improve their joint performance expression.

The concert was given on May 19, 2016 in the Sogakudo Concert Hall at the Tokyo University of the Arts, with a program that spanned the fourth and fifth movements of Franz Schubert’s Piano Quintet in A Major (D667, “Trout”). The polished performance delighted audience and Scharoun Ensemble members alike — and hopefully our AI system was pleased, too!

After the concert, Isao Matsushita, composer and Vice President of the Tokyo University of the Arts, had this to say: “AI is on the cutting edge of scientific technology, and, in the near future, it may become indispensable in our daily lives. These days, you hear much about AI “winning” or “losing” against human beings, but what we saw here today suggests that we may be able to experience the joys of living with AI. It is precisely because the music we heard was performed by human beings that today’s performance shows we can share passion and performance together with AI.”

“I enjoyed this challenging joint performance,” added Scharoun Ensemble member Peter Riegelbauer. “I feel it is really exciting to be tackling the challenges of doing things that are thought to be impossible. One should try these activities not just in classical music but also in electronic music and in other genres. These initiatives are just beginning, but just as today’s technological environment is completely different from 20 years ago, 20 years from now, the environment will certainly be quite different from today.”

Since 2015, Yamaha has participated as a member of the Tokyo University of the Arts Center of Innovation (COI). This collaboration was part of these activities and is aligned with our firm commitment to the music and instruments of tomorrow.