Augmented Reality Makes Musical Entrance at This Year's Grammys

Innovative augmented reality technology creates avatar emulating blues musician who will "perform" for Grammy Awards attendees.

Spencer Chin, Senior Editor

March 25, 2022

3 Min Read
mew_kv_2 (002) (002).jpg
Augmented reality technology created by MEW enables the creation of avatars from images or videos with life-like renditions of motions as well as personal effects such as clothing.Image courtesy of MEW

This year’s Grammy awards in early April will have more than the usual performances by nominated awards winners and the pomp and circumstance of the awards ceremonies. Thanks to innovative augmented reality technology by a young company called M Entertainment Works (MEW), those fortunate enough to attend the festivities live will be to witness a virtual performance by legendary Blues musician Robert Johnson, who has been recreated as a 3-D avatar.

Traditional motion capture technology in 3-D presentations uses specialized 3-D motion suits. MEW has developed algorithms that stitch elements of existing images and videos of the desired subject together to create 3-D motion sequences.

The augmented reality technology goes beyond the subject’s body language during action sequences to their personal effects as well. “We have the technology to use the subject’s clothing and put it in the avatar, and create additional clothing as well,” said Taku Kodaira, CEO at MEW, during a recent interview with Design News. “When the avatar is moving, so is the clothing as the simulations create a stream of movement.”

Blues musician Robert Johnson was chosen to be the first subject the technology was developed to demonstrate, as he was a favorite of MEW’s director, Hitoshi Miyata, who is also a musician. Johnson is considered the “King of Delta Blues” and influenced other blues musicians, as well as the Rolling Stones and Eric Clapton. Johnson wrote 29 songs and died at the age of 27, so there were few images of him performing.

Related:VR System Lets Users “Touch” One Another in the Metaverse

Interactive Blues Musician

MEW’s augmented reality technology tries to overcome that limitation through the magic of augmented reality. At the Grammy Awards, guests can exclusively experience the show through iPads in a lounge area. Downloading an accompanying app,  the guests will be then able to view Robert Johnson playing songs and interacting through their smartphone. 

Kodaira hopes to offer the virtual experience to participants through a nominal fee, available through blockchain, that he considers analogous to purchasing tickets for a movie or concert. He thinks a marketplace could eventually be set up where users could exchange and trade the “ticket” to view the virtual experiences, with a small transaction fee, to help monetize the development and use of the technology.

According to Kodaira, MEW’s company partners spoke to Johnson’s family to get permission to feature him in the technology promotion at the Grammy ceremony. He added that while no other musician has so far been recreated with the technology, the possibility is there in the future.

Related:Advanced Home Tech for Viewing Super Bowl LVI

While the augmented reality technology demonstration was set up to illustrate a musician that is known in some circles, Kodaira said one reason the technology was developed is to help somewhat lesser-known artists, such as dancers whose careers typically fade in their ‘30s due to the physical demands of the profession. Kodaira’s wife is a dancer.

Kodaira suggested that dancers can use the technology to illustrate themselves in motion and create online performances that might eventually be monetized. Kodaira added that the sports industry has expressed interest in this technology.

Beyond entertainment, Kodaira believes the motion capture technology could be adapted for uses such as medical and industrial training.

 

About the Author

Spencer Chin

Senior Editor, Design News

Spencer Chin is a Senior Editor for Design News, covering the electronics beat, which includes semiconductors, components, power, embedded systems, artificial intelligence, augmented and virtual reality, and other related subjects. He is always open to ideas for coverage. Spencer has spent many years covering electronics for brands including Electronic Products, Electronic Buyers News, EE Times, Power Electronics, and electronics360. You can reach him at [email protected] or follow him at @spencerchin.

Sign up for the Design News Daily newsletter.

You May Also Like