Microsoft Says 'Mixed Reality' Is the Future, Not VR and AR

If Microsoft has its way reality won't be virtual or augmented, it'll be mixed. Whereas virtual reality (VR) creates entirely computer-generated environments and augmented reality (AR) overlays computer-generated imagery onto the real world, mixed reality (MR), a term dating all the way back to a 1994 whitepaper based on research conducted at the University of Toronto and the Nara Institute of Science and Technology in Japan, seeks to blend the physical and digital worlds together. In a design setting for example, imagine being able to use a real screwdriver to make adjustments to a virtual product design, or being able to overlay virtual additions and adjustments onto a physical prototype and you start to get the idea.

With so many competitors ranging from Google, HTC, and Facebook-owned Oculus already tackling VR and AR from all sides, Microsoft is introducing a new strategy to position itself as a key player in bringing these technologies into the enterprise and consumer space. At its Build conference earlier this month Microsoft announced it would begin taking preorders on Mixed Reality Development Kits targeted at letting OEMs develop their own MR headsets. This all comes in conjunction with MR capabilities already built into Windows 10 as well as Microsoft's already available HoloLens AR headset that has been specifically targeted at enterprise.  The first of these MR head-mounted displays (HMDs), is being developed by Acer. 

“We're really creating an ecosystem and it's not dissimilar to how people use computers today,” Greg Sullivan, Director of Communications for Windows and Devices at Microsoft, told Design News. In essence, Microsoft wants to make VR and AR tools as commonplace for enterprise users as a keyboard or mouse.

During his keynote at the Build conference, Alex Kipman, Technical Fellow of the Operating System Group at Microsoft, and the man credited with inventing the HoloLens, said mixed reality should completely replace VR and AR in the cultural lexicon. “[VR and AR] are not separate concepts. These are just labels for different points on the mixed reality continuum,” Kipman told the audience. “This is why, to simplify things, we call all of it 'Windows Mixed Reality.' ”

In addition to Acer, Microsoft has already announced deals with big-name computer hardware manufactures including Lenovo, Dell, and HP. And if the company's gambit succeeds Microsoft could leapfrog it's competitors, particularly in the enterprise space where the HoloLens has a firm hold but is perhaps being held back by its steep $3,000 price point.

“The cost piece is something we've worked hard to address, Sullivan told Design News. “It is true today to have an immersive experience you need a headset that is not exactly an impulse purchase and a really powerful PC.”

Sullivan said that Microsoft is looking at HoloLens as a cornerstone for the experience it wants to deliver, but at a more affordable cost. Without any major products on the market yet it has already become clear that the next step for VR, AR, MR, or whatever you'd like to call it, will be providing an untethered experience – headsets that offer inside-out tracking and are able to follow the wearer's movements without the need for wires and external sensors.

“We wanted to build on the learning that we achieved with HoloLens,” Sullivan said. “One of the hardest things to make these [HMDs] compelling is to do positional tracking of the headset itself with six-degrees of freedom (6DoF). We've taken the inside-out tracking we built for HoloLens and brought it to these affordable headsets our partners are making.”

Sullivan said Microsoft is aiming to create an ecosystem with its partners that is open, but has a degree of structure – allowing OEMs to iterate and innovate their own products will still maintaining a standard of performance. “ That notion of an open but structured ecosystem is one we're familiar with,” he said. “All of the HMDS from our hardware manufacturer partners will devliver HMDs that contain inside-out, 6DoF tracking. It is a requirement of our platform to have that because the quality of experience is contingent on it. If you have a headset that does not deliver [6DoF] we know over time people will be uncomfortable with the experience.”

With its upcoming Creator's Update for Windows 10, scheduled for the fall of 2017, Microsoft, is implementing MR support into Windows 10 for developers. A big part of this update is that it will allow MR applications to run on PCs with integrated graphics, meaning users won't need to spring for the latest, high-end GPU in order to run a MR headset. “You'll be able to connect them to a mainstream PC that costs closer to $500,” Sullivan said.

The operating system is also key to Microsoft addressing the next major obstacle -- complexity. “The number one reason that existing devices get returned is complexity of setup,” Sullivan said, adding that the Creator's Update allows MR headsets to run with plug-and-play functionality -- again, just like a keyboard, mouse, or other standard peripheral.

But even if you can solve the cost and complexity issues, the greatest challenge is still to convince enterprise customers that adding MR to their workflow is a worthwhile investment. Sullivan said companies using HoloLens are already seeing a dramatic return on investment and Microsoft believes the same thing will carry over for MR. He cited German industrial company ThyssenKrupp AG, which is using HoloLens to assist elevator repair technicians. “ThyssenKrupp elevator uses HoloLens to dramatically scale the expertise of its repair technicians and experts to a field of other technicians. And they could do that right outside of the box just by using the Skype client.” He also talked about the value proposition HoloLens has brought to other maintenance and repair tasks as well as training. “If you're teaching a mechanic to work on a jet engine, for example, it's much more efficient to teach them on a holographic engine than to remove a physical one from a jet and fly everyone to a specific location to for hands-on learning.”

At the Microsoft Build conference augmented reality company Scope AR announced it has added HoloLens support to its WorkLink software for creating AR and MR “smart instructions” for training and remote workers. "Through its built-in depth camera and sophisticated tracking technology, HoloLens is one of the best wearable devices for hands-free use of the MR applications so many organizations are seeking," said Scott Montgomerie, CEO of Scope AR, said in a statement from the Build conference "With WorkLink now supporting the device, enterprises finally have a simple and easy way to digitize visual instructions at scale -- empowering all workers to become their own expert -- and begin to realize the full potential of MR." 

Early reactions to Microsoft's new strategy have been varied, with some saying the company is making the landscape unnecessarily confusing. Early reviews of Acer's MR HMD have been mixed, with reviewers expressing confusion on how the headset differentiates itself from any other VR headset, given that it has no external cameras for taking in real-world imagery and input.

However, at a $399 cost, and with inside-out tracking capabilities, the Acer headset is does have a bit of a leg up on competitively priced products like the Oculus Rift and HTC Vive that require external tracking hardware. With other manufacturers yet to debut their version of a MR headset it's possible someone may create one with external cameras - essentially a budget version of the HoloLens. If Microsoft's MR headsets can beat big names like Oculus, Vive, and Google to the next-generation of headsets it could indeed position Microsoft to extend its already strong foothold in the enterprise and consumer PC market one step further.

"All of these other immersive HMDs are primarily used for gaming and media content,” Sullivan said. “We see HoloLens and the MR HMDs as part of the same broader platform – the mixed reality platform. We think over time you'll begin to see more productivity scenarios and use cases. And potentially over time you'll see the ecosystem fill out across that whole spectrum with a range of use cases and device types.”

Chris Wiltz is the Managing Editor of Design News.