The New Isaac Sim Makes Smarter Robots Using Smarter Simulation

Nvidia has said the latest release of its Isaac Sim platform will enable more sophisticated training of robots and collaborative robots (cobots). Isaac Sim, which also includes an SDK, is a simulation program that allows for robots to be trained in virtual environments under simulated conditions. Rather than time consuming – and expensive – real-world training, engineers can train the algorithms that run their machines virtually, then port them into a physical robot to have it perform its task. Nvidia has touted Isaac, first announced in 2017, for its robustness – allowing developers to run hundreds, even thousands, of tests in a time frame that would be impossible in the physical world.

At the GTC Digital conference, Nvidia offered more information about the features expected in the latest version – Isaac Sim 2020.1, due to be released in may.

The platform has enhanced its focus on synthetic data generation – allowing engineers to create virtual scenarios that can mimic real world conditions. Pulling in real-world datasets for machine learning can be helpful, but can be limited when it comes to edge cases, situations the robot will need to respond to, but are less likely to be encountered. Reliable performance in edge cases becomes especially important in safety critical and cobot applications. If a robot encounters a situation it hasn’t been trained for or doesn’t expect, the results can be hazardous for humans.

Isaac Sim 2020.1 will feature a new Domain Randomization Toolbox that will allow for various aspects of a simulation, including lighting, colors, materials, and textures, to be randomized across an unlimited number of simulations without developers having to make manual changes to the simulation themselves.

It can also randomize environmental assets. In a simple example, a warehouse robot can be run through simulations where not only the lighting and other environmental conditions of the warehouse are varied, but also the placement of shelves, equipment, and other objects. When brought over into a physical robot, this will result in a much more capable and adaptable machine.

Nvidia has also introduced ray tracing into Isaac for the first time. Ray tracing is an intensive way of rendering computer graphics in a way that mimics how light moves and interacts with objects – rather than trying to imitate light by adjusting pixel values. The result is a much more photorealistic rendering than you would get with traditional methods. If you’ve awed at the look of some of the SFX in recent big-budget films like Avengers: Endgame you’ve seen ray tracing in action.

By supporting ray tracing, Isaac now allows robots to be trained in simulations with much more visual accuracy to the real world. LiDAR and RGB depth cameras, which Isaac Sim 2020.1 will be able to simulate, are both affected by materials. Photorealistic renderings of environmental materials will give engineers using Isaac a better understand of how the sensors on their robots are responding to their operating environment.

Finally, Isaac Sim 2020.1 will be offering support for more robots. Carter, the virtual use case robot that appeared in earlier versions of Isaac has been replaced by a new manipulator robot dubbed, Leonardo. Those using cobots like the UR10 from Universal Robots will find a lot more fidelity between Leonardo and their own physical system. The system will also offer more robust support of ROS, allowing engineers to more easily load software stacks developed in ROS into Isaac.

A preview version of Isaac Sim 2020.1 is currently available for download. The full release is expected in May.

RELATED ARTICLES:

Chris Wiltz is a Senior Editor at  Design News covering emerging technologies including AI, VR/AR, blockchain, and robotics.