In an interview with Design News, Hiren Bhinde, Director Product Management at Qualcomm, said the 845 is designed to offer much more powerful processing and power saving capabilities to deliver higher-quality immersion.
“On the Snapdragon 835, we introduced a visual-inertial odometry (VIO) algorithm,” Bhinde explained. “What that means is finding where you are in a room. Let's say you're in a dark room or blindfolded, there are two things you'd like to know: where you are in the room and what is the room like—where is the chair, table, where are the walls...Just knowing where you are is how VIO helps, and that's mostly tracking.” This comes in contrast to today's PC-based VR systems, which rely on external sensors to track a wearer's locations and movements.
Bhinde said that Qualcomm was able to expand on this tracking capability by integrating simultaneous localization and mapping (SLAM) technology into the 845 chip. Those that follow the autonomous vehicle space will recognize SLAM as the same technology that helps self-driving cars understand their position on the road in relation to other vehicles and objects by creating and updating a map of the vehicle's surroundings. Using external cameras on a headset, developers can take advantage of this same technology to track a wearer's position in a large space.
“From a standalone VR perspective, the way it usually works is you look at features in the room—things like the TV and chairs that aren't moving—and look at their relative position to triangulate the wearer's position,” Bhinde said.
“For SLAM, you also want mapping,” he added, explaining that Qualcomm developed its own proprietary mapping algorithms. “Why did we do that? Well, for VR today, when you map the room, you know where the boundaries of the room are. And virtual objects need to be registered or localized with the real world.”
Bhinde likes the example of two medical students being trained on a procedure in VR. Let's say both students are wearing augmented reality glasses and performing a procedure where virtual objects are overlayed onto the real world. If one student is holding a virtual scalpel, that object needs to appear in the proper place for both students without resetting or jumping around. The idea is to map these objects as well as the room itself to create an XR experience with perfect fidelity to the real world.
“For SLAM, there are three key aspects we looked at: the boundary system; the localization aspect; and then, of course, just being able to provide room-scale 6DoF,” Bhinde said.
|Qualcomm's new VR reference platform based on the Snapdragon 845 will allow engineers to develop around next generation mobile features, such as hand tracking. (Image source: Qualcomm)|
Both Bhinde and Swat said that Qualcomm looks at three pillars of immersion to provide the optimal XR experience: visual, audio, and interaction. Bhinde said the 845 is designed to excel specifically in these three areas.
“On the visual end, you have to make sure the GPUs in mobile devices can do complicated processing for things like shading, which means extreme detail as well as pixel quality and quantity,” he said. “Interaction means having computer vision capabilities like SLAM and VIO.”
It's one thing to get the algorithm right, but there's the other area of making sure you're able to sustain performance and that the device doesn't become hot. “Imagine running on a treadmill at high elevation with dumbbells in your hands and weights on your legs. That's what XR does to a processor,” Bhinde said, adding that Qualcomm has been able to overcome these power challenges with a combination of hardware optimization and algorithms.
XR Needs 5G and AI
But the proliferation of mobile VR isn't going to depend solely on high-performance hardware. High-end applications—particularly those on the enterprise end, such as collaborative design and virtual training—are going to require powerful network connectivity—especially for complex, multiuser applications.
Perhaps this is where the long-awaited 5G technology can come in. “XR is, if not the most important, a very crucial use case for 5G,” Bhinde said. “At the end of the day, if you're in London and I'm in San Diego and we're having a social VR/AR experience, it's going to require extremely high bandwidth with ultra-low latency.”
Bhinde added that Qualcomm feels artificial intelligence will form the foundation to any merger between XR and 5G. Smarter processors with the ability to learn over time and make predictions can help ensure the low latency needs for XR use cases in 5G.
“One of the key challenges is latency in VR/AR devices. So if I'm able to predict where my hands are going to be or where my head is going to move next, that prediction could help me display and render the next visual even better,” Bhinde said. “And those sort of predictive algorithms could be enhanced using AI. We do think that AI will help the devices learn what's happening next even better.”
While AI is used in XR applications today for things such as virtual training, Bhinde said the next phase of processors optimized for XR will incorporate this sort of functionality directly into the hardware. “In the times to come, we'll be seeing processors with the ability to learn and think smarter and more intelligently, and we'll be applying those capabilities for XR use cases and for low latency in our 5G pipeline.”
Chris Wiltz is a Senior Editor at Design News covering emerging technologies including AI, VR/AR, and robotics.
North America's Premier Battery Conference.
Join our in-depth conference program with over 100 technical discussions covering topics from new battery technologies and chemistries to BMS and thermal management.
The Battery Show. Sept. 11-13, 2018, in Novi, MI. Register for the event, hosted by Design News’ parent company UBM.