The popular notion of the go-anywhere, go-anytime, sleep-in-the-back autonomous car crumbled a bit in the last few weeks, as automakers admitted that the development of full self-driving technology is more difficult than expected.
Questions about the technology’s future reached full public view in April, when Ford Motor Co. CEO Jim Hackett acknowledged what had already become painfully obvious to much of the engineering community. “We overestimated the arrival of autonomous vehicles,” Hackett was quoted as saying by numerous news outlets. “Its applications will be narrow, what we call geo-fenced, because the problem is so complex.”
The admission came as a shocker to many in the public and in the media, essentially because it flew in the face of a growing belief that shiny new autonomous vehicles would soon be landing in dealerships.
Still, Hackett wasn’t the first to make such an admission. The auto industry had been scattering clues to that effect for months before the Ford statement. In November, 2018, for example, John Krafcik, CEO of Google’s self-driving car unit Waymo, had been even more blunt than Hackett. “It’s really, really hard,” Krafcik said during a live-streamed tech conference. “You don’t know what you don’t know until you’re actually in there and trying to do things.”
Waymo engineers are finding that full autonomy is “really, really hard.” (Image source: Waymo)
Krafcik went on to say that the auto industry might never produce a car capable of driving at any time of year, in any weather, under any conditions. “Autonomy will always have some constraints,” he added.
The comments by Krafcik and Hackett reinforced what many industry analysts had been saying for more than two years. “I agree with John Krafcik’s comment,” noted Sam Abuelsamid principal analyst for Navigant Research, which publishes an extensive annual assessment on the state of automated vehicles. “There’s no guarantee that we will ever have automated vehicles in the foreseeable future that are capable of operating everywhere, all the time.”
Level 5 Dilemma
Such statements, of course, contrast sharply with earlier declarations. Only three years ago, many OEMs, buoyed by advances in robotics technology, boldly foresaw a day in the not-too-distant future when drivers would be considered a redundant component. Ford, for example, predicted it would happen as soon as 2021. “There’s going to be no steering wheel,” former-CEO Mark Fields said in 2016. “There’s not going to be a gas pedal. There’s not going to be a brake pedal and, of course, a driver is not going to be required.”
Nor was Ford alone. Honda had publicly discussed having driverless cars on the streets of Tokyo in time for the 2020 Summer Olympics. Volvo, Hyundai, Daimler, Tesla, Fiat Chrysler, Renault-Nissan, and others were aiming for dates ranging from 2018 to 2025. Some talked of limited capabilities: highway driving first, for example, followed later by urban capabilities. But the message was essentially the same: The future was on our doorstep.
In 2016, Ford boldly predicted it would have vehicles without steering wheels, gas pedals, or brake pedals in 2021. (image source: Ford Motor Co.)
To be sure, the message isn’t dramatically different today. An autonomous future is still out there. But the rhetoric is being toned down. Most OEMs are now more forthright about the fact that autonomy will be a succession of small, graduated steps. First, there will be across-the-board, automatic emergency braking. Then robotic package delivery and robo-taxis in geo-fenced areas. Drivers will initially sit with their hands near the steering wheel, then they’ll move away. High automation – so-called SAE Level 4 driving – will arrive for prescribed locations. But the Holy Grail of autonomy – full SAE Level 5 go-anywhere, go-anytime driving -- is now recognized as more complex.
The reasons for the complexity are many. First, there’s weather. Industry insiders say it’s no coincidence that the most prominent autonomous test programs are located in California, Arizona, and Nevada, rather than Maine or Minnesota.
“Snow is difficult for a variety of reasons,” Stewart Sellars, general manager of the LiDar Group for Analog Devices, Inc., told Design News. “Most of the sensors we use in autonomy rely on line of sight. You’re using cameras, LiDar, or radar, and snow is basically an occlusion. It blocks the ability for those sensors to get their signal back.”
And it’s not only a matter of snowflakes fluttering through the air, blocking the return signal. Snow also tends to pile up on roadsides and medians, obstructing the road markings that are so critical to autonomous lanekeeping.
Moreover, it’s not just snow. Different geographies present a variety of weather challenges. “If you go to the Northeast, and you have ice and heavy rain and hail, there’s a very different problem set to solve,” Sellars said. “So, yes, it could take longer than perhaps people expected.”
The Testing Challenge
Perhaps the biggest technical obstacle, however, is converting human understanding into robotic intelligence. The intelligence that enables human beings to drive a car is largely taken for granted, and replicating it is proving to be a bigger chore than engineers foresaw.
“If you think about it, when you’re driving on the road, you’re dealing with hundreds of use cases for every mile you drive,” Sellars said. “You’re seeing things, and you know intuitively how to respond.”
And while those “use cases” might seem simple to human drivers, they’re not so simple to machines. When a piece of cardboard blows across a roadway 200 yards ahead, for example, human drivers quickly determine whether they should run over it or veer around it. Not so for a machine. Is it a piece of metal? Is it heavy or light? Does a machine even “know” that a heavy chunk of metal doesn’t blow across the roadway? It’s a tougher problem.
Most such problems need to be dealt with through test – either by driving physical miles or performing software simulations. Both approaches have their place, largely because software simulations can’t foresee every eventuality. When a car arrives at a four-way stop at the same time as another vehicle, for example, it’s a dilemma for a machine. Human drivers tend to nod or make eye contact, but microcontrollers can’t do that. Some vehicle developers are now teaching vehicles to inch forward while monitoring the other vehicle for implied consent, but such situations aren’t simple, and typically can’t be simulated today.
Makers of simulation products are working on that, and successfully broadening the amount of test that can be performed in software. Today, there are essentially two ways to simulate, experts say: first, by recording real-world events and playing them back in software and, second, by enhancing those playbacks to include situations that hadn’t been recorded.
“Our view is you need both,” noted Wensi Jin, automotive industry manager for MathWorks, which makes a software product called Automated Driving Toolbox. “Human imagination is limited and there will always be real-world cases that you can’t imagine. So you have to be able to take a certain amount of data from the playback and replicate it in a simulation environment, so you can do what-if studies.”
For suppliers and OEMs alike, such procedures represent a brave new world of test and validation. Suppliers say that the process is a departure from all test procedures used prior to the advent of the autonomous car. For the autonomous car, they say, it’s no longer enough to provide a part that meets a prescribed specification. Suppliers now have to help their OEM customers understand sensors and algorithm development, all in the context of use cases, rather than simple specs.
“It’s not like you’re just supplying an airbag sensor that has to meet a specification,” Sellars said. “With an autonomous car in an unconstrained environment, you have to think about all the use cases. That’s the biggest challenge, and you can only solve it with massive amounts of testing.”
Indeed, countless hours of test are required. A big part of that stems from the fact that engineers “don’t know what they don’t know,” as Waymo’s Krafcik has said. They need more test hours, essentially to account for the use cases they can’t imagine. As a result, most experts are estimating that the number of test miles needs to be measured in the billions. Toyota, for example, has publicly stated it needs 8.8 billion test miles for safe deployment of self-driving vehicles.
Whatever the figure may be, however, virtually everyone agrees that a large amount of physical testing is still inevitable. “There are certain situations we can’t simulate because it involves human behavior,” Sellars said. “So the number of physical miles driven is going to have to be a huge, huge part of this.”
Automakers Go Stealthy
For automakers, the big problem in all this is money. Manufacturers are spending vast amounts of cash in their autonomous development programs, and constantly looking to investors to raise more. GM Cruise LLC, for example, recently announced an equity investment of $1.15 billion from a group of institutional investors. The new funding brought the company’s valuation to a stunning $19 billion – about a third of the total value of General Motors Corp. Cruise plans to use the money to double its workforce and triple its San Francisco-based office space.
In 2016, General Motors invested approximately $600 million in Cruise Automation’s robotics expertise. (Image source: General Motors)
Such figures aren’t just confined to GM, however. Most of the industry is simultaneously burning through cash. Ford, for example, invested $1 billion in Argo AI; Toyota put $1 billion in Toyota Research Institute; GM invested $500 million in Lyft,Inc.; Volvo entered into a $300 million joint venture with Uber Technologies Inc., and Intel is said to have spent $15.3 billion to acquire Mobileye.
Automakers say it’s unlike anything they’ve seen before. “It’s the most engineering-intensive thing ever attempted,” said one automotive executive in an off-the record discussion with Design News. “And you need lots of the world’s best engineers to do it. I’m not talking about tens or hundreds of engineers. It’s in the thousands. We’re talking about billions of dollars.”
That’s why some companies are now backing away from near-term projections, the executive said. They’re seeing the amount of test needed, the engineering requirements, and the cost, and they’re wondering how long this will take.
"The auto industry is getting scared, like when you're practicing for something and then you have to get on stage and actually do it," noted Mike Ramsey, senior director and automotive analyst for Gartner, Inc. "And then you suddenly realize, 'Maybe I'm not as prepared as I thought I was.'"
To be sure, however, not every automaker is pushing the timetable back. Tesla Inc. CEO Elon Musk has maintained his belief that his company will have full autonomy in 2020. “My guess as to when we think it’s safe for somebody to essentially fall asleep and wake up at the destination – probably towards the end of next year,” he said in a February podcast. More recently, he doubled down on that statement, saying he plans to have more than a million robo-taxis on the road in 2020. The key, he said, is the fact that Tesla can more effectively test its autonomous driving technology because it accumulates “100 times more miles per day than everyone else.”
Privately, most industry engineers doubt Musk’s claims. But they’re saying little, preferring instead to remain in stealth mode. Some hint at a Level 5 arrival in the late 2020s or early 2030s. But, in general, automakers say they’re no longer in the prediction business.
“We knew from the beginning it was hard,” one engineer told us. “That’s all we’re saying.”
Still, virtually every automaker and supplier is forging ahead at full throttle. “It’s inevitable,” Sellars told us. “It’s going to happen. The only question is how long it will be before we can walk into a dealership and buy a Level 5 car.”
Senior technical editor Chuck Murray has been writing about technology for 35 years. He joined Design News in 1987, and has covered electronics, automation, fluid power, and auto.
Learn about the economic and regulatory forces that are making it necessary for long-time rivals to team up in pursuit of their autonomous goals at the technical session, Partnerships Will Be More Important Than Ever for Automated Driving. The technical session, hosted by Sam Abuelsamid of Navigant Research, will be offered at Drive World Conference & Expo, which takes place from August 27-29. The inaugural three-day showcase brings together the brightest minds across the automotive electronics and embedded systems industries who are looking to shape the technology of tomorrow.