BMW needs to amass 150 million virtual miles of testing for its autonomous driving technology before it will be ready for customers, so the company is relying on Unity’s real-time 3D tools to produce the virtual world where it can test robot cars in complete safety.

Indeed, BMW does 95 percent of its autonomous vehicle testing in simulations rather than on the road. This means the company can do more testing in less time, in more conditions, for less cost, and at zero risk.

There’s no need to wait for darkness, fog, rain, or snow in the virtual world, where all of these things can be tested in infinite variations. And the very same traffic challenges can be presented under each of these different conditions, gauging the autonomous pilot’s success at avoiding a collision after being cut off in traffic not only in perfect conditions but at night and in bad weather too.

BMW does this work at its Autonomous Driving Campus in Unterschleissheim, Germany, just north of Munich.

“At BMW, we believe simulation is key for developing autonomous driving,” says Nicholas Dunning, Simulation Expert for Automated Driving Functions, BMW Group. “Unity plays a pivotal role in helping our team create, visualize, and evaluate the millions of virtual road trips needed to help us achieve our AD ambitions.”

Related:FLIR's Thermal Sensors Are Coming to Self-Driving Cars in 2021

Dunning works as part of the core 12-person development team that has built custom tools made with Unity to help the 1,800 autonomous driving developers at BMW’s campus visualize and advance their work.

BMW selected Unity 3D based on a number of benefits, reports Dunning. “We chose Unity 3D because of multiple factors, including previous work creating prototypes and proof-of-concept experiments within this environment as well as the ability to easily unit- and integration-test all of our code by clearly separating our code base into different layers of abstraction.”

BMW’s primary uses for Unity are to visualize raw data from simulations in an immediately understandable, true-to-life way, beyond graphs and charts, and then to evaluate the current state of the autonomous driving functions in a variety of simulated scenarios.

BMW used Unity to create a graphical editor that makes it easy to change test scenarios, with variations in the quantity and type of traffic, presence of pedestrians, traffic signs, the lay of the road, and its markings, as well as the environmental conditions for light and weather.

“Without our simulation tool, we would rely more heavily on other tools provided by the various communication protocols and middleware layers the Autonomous Driving code is built on,” explains Dunning. “This has the negative consequence that each time a change to these middleware applications is made, your tooling also has to change.”

One change BMW made was to step back from the highly detailed, realistic driving environments that BMW developers created initially. Those detailed environments proved to produce data noise, so developers simplified the scenarios to better concentrate on the outcome of each simulation.

In its small-scale tests, BMW’s teams log tens of thousands of miles of brief, short-distance tests that usually take less than a minute. This helps evaluate portions of the autonomous driving program, such as vehicle trajectory planning.

In the large-scale system tests, BMW plays out extended scenarios such as a drive from Munich to Stuttgart. Such tests take hours to run and incorporate moving vehicles, pedestrians, and variable weather along with map data, sensor input, and vehicle dynamics.

These tests can run all night long, letting BMW’s developers examine logs in the morning that describe when functions failed during tests. They especially focus on edge-case scenarios such as encountered a traffic vehicle that runs a stop sign, because the resulting near-collisions are difficult and dangerous to stage in the real world.

“With more simulation-based testing, the coverage of different situations greatly increases, leading to overall safer end product for our customers,” Dunning says. “Every kilometer driven in simulation one that does not need to be driven on the road, as you, as an Autonomous Driving engineer, are already testing the change you make right away.”

“This not only reduces the development time in terms of testing a specific change in an isolated environment, you also hit the stages of integration much earlier because you can already see this change working together in the more complex variants of the software stack.”

When errors, or “incidents,” occur during a simulated drive, BMW engineers can quickly identify them and address the causes. “Our Virtual Endurance Testing setup allows us to drive between various cities in Germany and automatically detect situations where certain [Kilometers per Incident) drop below their acceptable thresholds,” Dunning explains. “This could be the distance kept to other traffic, or the deactivation of the Autonomous Driving function to name just a few.”

Engineers follow up by analyzing the reports based on these measurements. “Smaller snippets of these longer drives can be automatically extracted and inspected by one of the engineers,” adds Dunning. “If the situation is deemed plausible, then this becomes the basis for a feature or acceptance test which protects against regression.”

It is all well and good to succeed in the virtual world, as we saw earlier this year when fans were left to watch race drivers competing on simulators rather than with their usual race cars. The question is how well those outcomes reflect reality.

“We have many mechanisms to ensure the validity of our simulation,” he continues, “Including cross-validation between recorded data and scenarios extracted from that same source as well as an equally strong push towards a combined strategy of re-processing that uses a changed version of the Autonomous Driving function the same inputs again and see how the perception of the situation has changed, in combination with simulation with more idealized inputs.”

This includes deploying simulation tooling all the way from pure Software-in-the-Loop all the way through to Hardware-in-the-Loop platforms including some hardware bits from the car to detect problems and trace them back to their root causes.

In the future, BMW would benefit from a few upgrades to Unity 3D, says Dunning. “We would like to simulate a wider range of situations from many different and unique driving locations, as well as pushing the efforts already being made to bring greater standardization to the greater simulation community.”

And to further improve the fidelity of the simulation with the real world, the company would like to be able to incorporate entire vehicles into the system rather than only some hardware components, he adds. “Another big step would be to move into Vehicle-in-the-Loop tests as well, and bridge the final integration gap to further improve the speed and effectiveness of the complete development cycle.”

Do you suppose test miles accumulated with a real vehicle in the loop count toward the 150 million virtual mile target, or do those go into the real-world test total?