With Isaac, Nvidia Trains Robots in Virtual Environments

Remember how the heroes in The Matrix were able to learn new skills almost instantly by training in a virtual environment? It might not be a realistic way for humans to learn (yet), but Nvidia believes it can already be done for robots.

Wednesday, at his keynote during Nvidia's GPU Technology Conference (GTC), Nvidia CEO Jensen Huang called for an “alternate universe,” where robots could learn but not be beholden to the laws of time. “This universe needs to train at warp speed,” he said.

The virtual environment, which Nvidia is calling Isaac (named after Newton and Asimov, according to Huang) is an alternative to real-world reinforcement learning, which has robots and AI learn in the same way humans do – through trial and error. On the low end reinforcement learning could teach a robot a simple assembly task. On the highest end you get an AI that can beat champion players at Go.

The problem with doing reinforcement learning in the physical world with robots is that it can be time consuming and expensive to wait around while a machine figures out how to do a task perfectly each time. It can also be dangerous if larger robots are involved. By doing all of the training virtually Nvidia says robotics companies can save significant amounts of development time. “The virtual brain gets trained, then transferred into a real robot, and the robot does its last bit of adaptation in the physical world,” Huang said. Essentially the robot would be ready to go about its task immediately after installation.

As a major player in the computer graphics card market, Nvidia is no stranger to creating high-end simulations with real-world physics. In fact, Isaac will leverage a modified version of the popular Unreal Engine 4 (the engine behind popular video games such as Gears of War, Street Fighter V, and Robo Recall) to create its virtual environments. Bob Pette, VP of Visualizations at Nvidia, told Design News that part of the advantage of using Unreal is that developers with a knowledge of the engine may also be able to modify it and create their own environments the same way that video game creators do. The system will also interface with the OpenAI Gym, an open-source toolkit of reinforcement learning algorithms, giving developers access to a library of already-existing training tools.

A demo video shown during the keynote showed a robot learning how to hit a hockey puck into a goal and then how to sink a golf put. Huang said the system can even run multiple simulations at once, then take the smartest one, replicate that, and create a new test group, continuously iterating and reducing the overall learning speed by many factors. And since its designed with deep learning, there is no programming involved, “the system tries until it figures out the task,” Huang said.

While a simple hockey demo is nice Jesse Clayton, Senoir Manager of Product Management, Intelligent Machines at Nvidia, told Design News that the same concept can be applied to factory automation tasks, as well.

“Factory robots are limited to fixed functions; they don't deal well with dynamic situations,” Clayton said. “It takes time to set them up to deal with new part. ... Factory robots deal well with large run items, but not small run. One of the benefits [of Isaac] is you can train the robot very quickly and leverage it for smaller run items.”

Nvidia hasn't yet determined any benchmarks or testing to how much time and cost could be saved with Isaac, but Clayton said the company is confident it can save months of training time. He added that the training time will also depend on the complexity of the robot. A smaller robot can move faster than a bigger one, for example, so it could run through tasks faster and learn more quickly.

Isaac is part of a larger strategy Nvidia is calling AI on the Edge, built around its new line of Jetson development boards. The idea is to lower the barrier to entry into AI robotics for companies by providing a development board (the Jetson), a series of reference platform robots, as well as a training simulator (Isaac) to get companies of all sizes up and running quickly.

Right now the initial focus is to target Isaac toward robotics researchers and to make it more generally available in the long term. The company has set up a site for those interested in learning more and signing up to try Isaac.

Chris Wiltz is the Managing Editor of Design News.