Autonomous Cars Will Require Years of Test

Testing and validation of autonomous cars is unlike anything experienced previously in the 130-year history of the automobile.

The reason is simple: Developing a self-driving vehicle that’s safe in every scenario means that engineers have to foresee, and then test for, every imaginable situation. As a result, they have to analyze vehicle performance with every means at their disposal—on test tracks, on public roads, and in electronic modeling and simulation. Most experts agree that the process could take a decade or more in order to reach complete (so-called Level 5) autonomy.

To learn more about the challenges facing the engineers who are doing the work, Design News sat down with Jeff Phillips, head of automotive marketing for National Instruments Corp., which supplies automated test equipment and virtual instrumentation. Phillips, who holds a bachelor’s degree in mechanical engineering from the University of Tennessee, is responsible for helping automakers deal with the challenges of validation and verification of next-generation vehicles.

DN: We keep hearing about the complexity and time involved in the testing of today’s vehicles because of all their electronic content. What makes the testing of today’s cars so much different from the testing of vehicles a few years ago?

Phillips: The way validation labs are set up now, you have certain car components that are electronic and certain components that are physical. And then you have software that runs on the ECU. And you have different vendors that serve each of those needs.

But nearly every automotive component today is all three of those things. So you have physical durability testing—opening and slamming doors 100,000 times. You have electronic measurements because there are sensors built into almost all the perimeter components of the car. And with all those sensors, you have semiconductor content, with an ECU that requires software validation testing. And you can’t test them independently because they are all inter-dependent.

So what’s happening is these three very distinct worlds have different workflows that are slamming into one another. And they’re putting pressure on the test departments that are used to low-innovation cycles.

DN: So how is autonomous different? Is it a special case?

Phillips: Autonomous is special because it’s duplicative of all the challenges individually. When you do validation, you’re doing it with emulated sensors and data. So now you’re taking emulated validation data with sensors, and you’re plugging that data back together for the integration test for the full car.

DN: So that’s the complexity. Where does the additional time come in?

Phillips: The time comes in because the upfront components are being innovated, rather than just iteratively improved. So whereas the sensors (in the past) may have just been one backup camera and one radar sensor on the front of the car for emergency braking, now you’ve got multiple cameras. You’ve got monochromatic cameras and high definition cameras, and they’re weighing on the processing capabilities of the ECU.

So the test cycle is compressed because the design cycle is cutting into the test time. It’s almost like being on vacation—whatever you were planning to do on the last day is always the thing you have the least amount of time to do.

DN: We’re hearing about all the possible scenarios that have to be simulated. Does this play into the time factor?

Phillips: It plays in tremendously. And the problem is we don’t have a regulation or standard we can follow. There is no standard set of use cases that say, “If you come upon this use case, here’s how your sensor and autonomous algorithm should react.” We leave it up to the discretion of the OEM.

DN: What can be done about the lack of standards?

Phillips: That’s the question of the century. We want to help drive the definition of those standards. Being on the forefront of that enables us to help our accounts to do tests and meet the standards. We want to show them the fastest way to simulate. What’s the right set of models? What’s the right set of scenarios in order to meet some future specification that lets us designate a car as a Level 4? It gives the public some baseline of understanding of what it all means.

DN: Some of the automakers are causing confusion by saying they will have autonomous cars by 2020, without really explaining what they mean. Is this a problem?

Phillips: Public ignorance is a major issue. In order to get to Level 5—in other words, a fully autonomous car with no steering wheel or pedals—we’re going to spend many years at Level 3 and Level 4. Many years. The data from those systems, and the inevitable accidents and crashes that occur, will serve as the data that defines the regulations for Level 5.

So for a long time, you’re going to have semi-autonomous, which means the car can drive itself in well-defined conditions. But there’s going to be a driver’s seat; there’s going to be a steering wheel; and there’s going to be pedals.

There’s still a lot to be worked out. There are scenarios where the car will have to break the law in order to proceed. One common scenario is, you’re driving down a two-lane highway—one lane each way—and there’s a Fed Ex truck in front of you, parked on the curb. You can’t go around it without crossing the double-yellow line. Are you going to allow the car to break the law? Now, you’re getting into a whole different set of rules, regulations, and even morality decisions.

That’s one of an infinite number of scenarios that still have to be worked out.

RELATED ARTICLES:

DN: Will OEMs be willing to let suppliers drive the discussion?

Phillips: They will if the suppliers are willing to take responsibility for the outcome of the systems. And that means the suppliers will then want to supply the entire system.

DN: So the relationship between the OEMs and suppliers will come down to who’s willing to take the responsibility?

Phillips: There’s a lot of corporate responsibility, culpability, and insurance matters that the OEMs and Tier Ones need to work out, primarily with local governments. It’s going to come down to who owns the algorithms for ADAS. Is it the Tier One? The OEM?

If it’s the Tier One, then they will only sell full ADAS instrumentation solutions packaged together. If it’s the OEM, then the Tier Ones will sell pieces of the solution.  

There’s a lot to be determined from a regulations and accountability standpoint, and that will harden the way the supply chain works together on this.   

Senior technical editor Chuck Murray has been writing about technology for 34 years. He joined Design News in 1987, and has covered electronics, automation, fluid power, and auto.