At the end of Iron Man
2, when the heroes and villains are flying, flipping, punching and rolling
about in a Japanese garden, the real stars aren't the actors. They're a bunch
of matchbox-sized sensor packs sewn into the lining of the so-called "mo-cap"
suits, which enable stunt people to perform physical feats that ... well, even
stunt people can't do.
The
technology that allows actors to fly off tall buildings or run through walls
has been has been available for several years, of course, but these new
sensor-laden suits are taking moviemaking to new heights.
"With this
technology, you can exaggerate everything," says Harvey Weinberg, group leader
of applications engineering for Analog Devices
Inc., maker of the sensors. "You can have an actor jump off a three-foot
platform and with a motion-capture suit and computer graphics, the platform
becomes a 10-story building."
Actually,
it does that and much more. The new breed of sensors and suits is transforming
the process of "previsualization" (or "pre-vis," as it's known in Hollywood lingo). Instead of artist-rendered storyboards
or black suits with luminous markers, directors can now use gyroscopes and
accelerometers to visualize scenes before filming them. It's a huge step
forward for movie makers, largely because it removes multiple cameras and sound
stages from the previsualization process. Directors can now do the pre-vis step
wherever they want - in a hallway, parking lot, or in someone's backyard.
"We can go
anywhere with this suit," says Chris Edwards, chief executive officer of The Third Floor, the largest
previsualization company in the world. "We don't need any special rooms with
lots of heavy gear. We can take it on a plane and go to a director's house in Europe and set it up with a laptop. We can do a remote
motion capture recording session there and get high fidelity data in real
time."
|
Moreover, the
new
MVN Motion Capture Suits,
as they're known, are bringing new capabilities to a broader swath of moviemakers.
Until now, such digital capabilities were reserved for the likes of James
Cameron and others with mammoth budgets.
"My hope is
that we're embarking on a golden age of cinematic content creation, where
finally these technological tools will enable inspired directors with average
budgets to create whatever their hearts desire," Edwards says.
Beyond
Optical
At first glance, the marriage of inertial sensors and Hollywood moviemaking would seem an unlikely one. Microelectromechanical
systems-based (MEMS-based) inertial sensors debuted in the early 1990s, first
in automotive air bags and stability control systems. Later, they began
appearing in smart phones, laptop computers and video games, such as
Nintendo Wii.
But as sensor performance improved
over the years and costs dropped, engineers began finding new uses for the
technology. The most popular of the inertial sensors - accelerometers - have
become increasingly appealing to engineers designing medical and industrial equipment,
as well as automotive systems. Tens of millions of the devices are now employed
every year to measure motion in linear axes. At the same time, MEMS-based
gyroscopes have added another dimension to inertial measurement, enabling
engineers to build products that can determine the rate of rotation about an
axis.
Xsens
Technologies B.V., a Netherlands-based supplier of 3-D motion-tracking
products, began researching the concept of a sensor-based suit for
biomechanical measurements in 2005. A 25-person research team, including about a
dozen Ph.D.s, was assigned to the project of applying accelerometers,
gyroscopes, digital processors and Bluetooth transmitters to the Lycra suit,
and then tying the hardware together with a laptop-based software model. By August,
2007, the company was demonstrating the technology at the
Siggraph computer graphics
conference.
"I went to Siggraph and they were
unveiling this killer app," Edwards says. "It was a way to do mo-cap (motion
capture) unlike anything I'd ever seen."
Indeed, until that time, Edwards
and others who did "pre-vis" thought the state-of-the-art was an optical method
that involved suits with luminous markers. The optical technique, however, contained
a level of complexity that was especially vexing for smaller companies.
"If you have an optical system, you
have to rent more space, use expensive cameras and hire trained operators,"
says Erik Wilbrink, vice president of sales and marketing for Xsens, "whereas
with our system, people are motion-capturing and getting real-time data within
15 minutes."
Collecting the
Data
But real-time feedback is just one of many advantages The
Third Floor got from the MVN Motion Capture Suit.
The suit,
which incorporates 17 matchbox-sized "trackers," enables directors to take a
set of data points and turn them into a scene full of characters, environments
and props that can be re-arranged with a laptop computer. Containing three
single-axis
iMEMS gyroscopes, two dual-axis
iMEMS accelerometers, and one
Blackfin
digital signal processor (DSP) from Analog Devices, each tracker is sewn
into fabric tunnels within the suit. Wires in the tunnels connect the trackers
in long daisy chains to an Xbus Master - a controller that synchronizes and
powers them while simultaneously sending data to a laptop computer through a
Bluetooth connection.
Essentially, the suit comprises a six-degree-of-freedom
motion-sensing platform. The two dual-axis accelerometers collect acceleration
data in the x, y and z planes, with one extra axis remaining for redundancy.
The three gyros measure rate of rotation about each of the axes.
During a movie
shoot, pre-vis experts gather data from the inertial measurement units (IMUs),
which are located in the middle of actors' body segments, such as mid-calf,
mid-forearm, mid-thigh and other spots around the body. Data from those points
is sent to a biomechanical software model in the laptop, which tallies motion
data at each point in real time. Computer graphics imagery (CGI) software then
superimposes characters, objects and environments over the motion data.
"The
sensors give us acceleration data and orientation data," says Hein Beute,
product manager for the entertainment market at Xsens. "When you feed that data
into a biomechanical model, then you know the orientation of the upper arm and
lower arm and legs. Within a millisecond, you can re-construct an actor's
pose."
During
motion capture, the sensor-based suits offer two important advantages over
optical techniques: They can gather data over huge areas and they eliminate the
problem of "occlusion," which occurs when one body part blocks another,
preventing the transmission of data. Because the MVN suit transmits data by RF,
rather than optically, occlusion is not an issue.
"When we
tell people they don't have to deal with occlusion anymore, there's a big sigh
of relief," Wilbrink says. "With occlusion, there's missing data. And the only
solution to that is to add more cameras, which makes it more complex."
Living Story
Board
Engineers at The Third Floor employed the MVN suit for
Iron Man 2's action sequences and fight
scenes. The sensor-based technology was especially important for a complex
20-minute scene known as the Japanese garden sequence near the end of the
movie, which involved multiple characters, props and environments, they say.
"We had
bi-pedal humans and robotic characters, and they were supposed to be running
around and punching and flipping each other," Edwards says. "Within very little
time, the actors got into their motion capture suits and did the fight
choreography, punching and flipping to their hearts' content." The director's team
then compiled single elements of the battle into a larger scene that involved
integration of multiple clips.
The Third
Floor used a similar approach in another recent movie,
Alice in Wonderland. There, engineers used the
MVN motion capture suits on actors running on treadmills, then took the data
and later "composited" the actors' movements into a virtual world.
"If you can
put a lot of detail into your pre-vis, which is what we always try to do, then
directors treat it more seriously," Edwards says. "All of a sudden, what you've
built is more than a story board. It's a living, breathing design document for
each and every shot."
Engineers
from Xsens say they foresee the technology being employed in biomechanical
research, movement science, ergonomics, human factors design, sports analysis
and virtual reality applications. Computer games are also a growing market for
the technology. Gearbox Software employed it for character animation in the
Borderlands video game, and Sony
Computer Entertainment /Guerilla Games used it for
Killzone 2. Moreover, the MVN suit is capturing the interest of a
growing number of Hollywood studios. In
addition to
Iron Man 2 and
Alice in Wonderland, it's been used in
movies made by Industrial Light & Magic, Sony Pictures Imageworks and
Double Negative.
"It's
amazing to see the directors' and cinematographers' eyes light up when they
realize, ‘You mean we can get in the suits and act this out right now?'"
Equally
important, though, is the effect the suit could have on smaller film companies
with tighter budgets.
"That's the
most exciting part," Edwards says. "With this, you don't need to be a
super-elite filmmaker to take advantage of the best tools."
Iron Man: © 2010 MVLFFLLC. TM & ©
2010 Marvel Entertainment. All Rights Reserved.
Tweet This
0 
