Watch ABB's YuMI Robot Conduct an Orchestra

YuMi the dual arm collaborative robot released by ABB had a night at the opera on earlier this month – as a conductor.

In a display that was part robotics demonstration, part art project YuMi conducted the Lucca Philharmonic Orchestra at the Teatro Verdi in Pisa, Italy, as part of the gala of the First International Festival of Robotics. The team at ABB set out to show that YuMi was capable of learning the precise movements of even an orchestra conductor.

Over the course of a few days engineers at ABB worked alongside Andrea Colombini, director of the Lucca Philharmonic Orchestra to teach robot the precise movements it would need to conduct an orchestra through a performance of "La Donna è Mobile" from Verdi's Rigoletto. Via a method called “lead-through programming” the robot taught the motions via hand-guidance by Columbini. Once the movements were recorded, the engineers were further able to refine them by having the robot rehearse with the orchestra's musicians.

The Director Meets the Engineers

Colobini worked with YuMi to make sure the arm and baton movements coordinated with the music. These pre-programmed movements enabled YuMi to effectively lead the orchestra. "The director, Colombini, was very involved with the motions and the timing involved," Nicolas De Keijser, the Assembly & Test Business line manager at ABB, told Design News.  "The conductor determined YuMi's movements ahead of time."

Colobini himself was encouraged by YuMi's capabilities. “YuMi’s sophisticated technology excited me and revealed many possible ways to realize art and music through the robot,” Colombini wrote in a blog for ABB. “Setting up the interaction between the elbow, forearm and wrist of the robot, making use of its versatility in repeated and demanding attempts to break down the upbeats and downbeats, was very successful.”

An Ease-of-Use Demonstration

According to De Keijser, the big overarching lesson in this display of YuMi capability is the ease of use and ease of programming. "Robots are not programmed to do fluid motion. In two days’ time, we were able to do it with a conductor who didn’t know anything about robots," said De Keijser. "The engineer would never have been able to come up with these motions. They required the conductor. Yet in two days, he was able to coordinate all that in a demonstration of ease of use and programming."

In a video released by ABB (below), Colombini suggested that a robot like YuMi could be useful for rehearsals, where the conductor cannot always be with the orchestra. Of course, the system is not a form of artificial intelligence able to learn how to adapt and conduct music on its own. It doesn't carry the sensitivity or creativity that a human conductor would have (so our human orchestras are safe ... for now).

What YuMi is doing here is akin to the same precise and repetitive tasks it would learn and perform on a factory floor. “While this performance gives an inspiring peek at the future, it is unlikely robots will ever prove capable of combining the scholarship, artistry, technique, interpretation and charisma of a skilled human conductor,” ABB said in a statement.”The simple goal is to develop industrial robots that are easier to use and perform better with less human intervention.”

Watch ABB's YuMi conduct the Lucca Philharmonic Orchestra:

Chris Wiltz is a senior editor at Design News covering emerging technologies, including VR/AR, AI, and robotics.