HOME  |  NEWS  |  BLOGS  |  MESSAGES  |  FEATURES  |  VIDEOS  |  WEBINARS  |  INDUSTRIES  |  FOCUS ON FUNDAMENTALS
  |  REGISTER  |  LOGIN  |  HELP
Rob Spiegel
User Rank
Blogger
A software glitch after all
Rob Spiegel   11/8/2013 8:39:53 AM
NO RATINGS
Nice story, Chuck. Perhaps the most telling line comes at the very end: "Toyota could have done this (implement a brake-throttle override system) in 2002 without any extra cost to the vehicle." 

tekochip
User Rank
Platinum
Re: A software glitch after all
tekochip   11/8/2013 9:47:09 AM
NO RATINGS
If they made the change a jury could have seen this as demonstrating culpability, suggesting that the change was made because Toyota was aware of a system flaw.  Unfortunately, not fixing known flaws is a common theme because of litigation fears.


Rob Spiegel
User Rank
Blogger
Re: A software glitch after all
Rob Spiegel   11/8/2013 10:33:50 AM
NO RATINGS
I never realized that change to improve a flaw is a form of admission of guilt. Wow. That's one strange moral conundrum.

jhankwitz
User Rank
Platinum
Re: A software glitch after all
jhankwitz   11/11/2013 10:05:58 AM
NO RATINGS
Companies have avoided documenting or taking corrective action for many years due to litigation fears. When working with companies back in the early '90s, many companies refused to become ISO 9001 registered because the standard required documentation of all actions taken to improve or correct anything. That requirement was subsequently removed from later versions of the standard.  I knew of several companies that actually shreaded all customer complaint and feedback documentation after reading it.  - John

Rob Spiegel
User Rank
Blogger
Re: A software glitch after all
Rob Spiegel   11/11/2013 1:05:58 PM
While I can understand a company's desire to keep problems with their products private, it puts its customers at risk. The company's well being is put at a higher priority than the safety of the customer.

naperlou
User Rank
Blogger
Re: A software glitch after all
naperlou   11/8/2013 11:26:55 AM
NO RATINGS
Software development for automobiles is a definitely a safety critical area.  This became crirical with drvice by wire.  I had a 2002 car that had an electronic engine management system, for example.  This was before "full" drive by wire.  When the EMS started to go, I was on my way home on the Interstate.  I could still drive the car, however.  It just ran rough. 

With full drive by wire, we should be using methods and techniques used for avionics and other safety critical systems.  This is becoming the case, but we will have a large overhang of vehicles, such as the Toyotas, that do not meet these standards.  Considering that any programming is amortized over a large number of units, adding this safety critical approach should not be costly.

Rob Spiegel
User Rank
Blogger
Re: A software glitch after all
Rob Spiegel   11/8/2013 11:56:07 AM
NO RATINGS
Apparently, though, adding safety measures after the fact is an admission of guilt, a signal (and effectively proof) that the earlier vehicles are not safe.

ttemple
User Rank
Platinum
Re: A software glitch after all
ttemple   11/8/2013 12:07:38 PM
As a software engineer, I have always believed that this would prove to be a software problem.

I think that at some point there will be incontrovertible evidince that code is/was causing the problems.

A faster "admission of guilt" will prove to have been cheaper in the end for Toyota. Had they accepted early on that there may be a problem in the code, they would have vigorously worked at identifying the problems, and perhaps have recalled affected vehicles and applied appropriate fixes.

The truth is/was bound to come out, so why not own up and get rid of the potential of cascading liability?

Rob Spiegel
User Rank
Blogger
Re: A software glitch after all
Rob Spiegel   11/8/2013 12:42:20 PM
NO RATINGS
I agree with you Ttemple. Unfortunately, they seem to have put up the barracades. This seems to be a common corporate problem. I think a chance in this behavior would require a change in corporate culture.

naperlou
User Rank
Blogger
Re: A software glitch after all
naperlou   11/8/2013 12:08:30 PM
NO RATINGS
Rob, that may be the case, but there is no ignoring the problem.  Once it has come to light it  needs to be fixed in future vehicles and, if possible, retrofitted into older ones. 

As far as the liability, if the company knew they had this problem and did nothing, or did an inadequate job of fixing it, then they have a liability problem no matter what.  The other issue is the standards at the time of sale.  I don't know, but I doubt that there were government standards that applied at the time.  That would be the key.

This is very reminicent of the safety stadards we see in cars today.  We have the crash test ratings, which did not exist before.  Early SUVs, you might recall, had much less crash worthiness than regular automobiles when the tests were first applied to them.  It took a lot of enginerring to get them up to standard.  SUVs are basically pick-up trucks with a different body.  Pick-up trucks never had the same standards as cars.  Now that we have had some of these problems, I think we will see a big move into standards.  As for the older vehicles, you take your chances.

Rob Spiegel
User Rank
Blogger
Re: A software glitch after all
Rob Spiegel   11/8/2013 12:45:43 PM
NO RATINGS
Yes, I remember that the early SUVs were effectively trucks, and their categorization as trucks gave these vehicles a pass on a number of goevernment standards. That was also true with the early minivans. 

Charles Murray
User Rank
Blogger
Re: A software glitch after all
Charles Murray   11/8/2013 5:33:51 PM
NO RATINGS
Good point, naperlou. If automaker knowingly rejects the idea of a fix, then that's a liability, too.

dhyoung
User Rank
Iron
Re: A software glitch after all
dhyoung   11/11/2013 11:05:02 AM
NO RATINGS
Does any of this conjecture respond to the notion than the application of full throttle cannot be overcome by the cars ordinary brake system? My cars engine is significantly more powerful than any Toyota. If I apply full throttle and full brake to my car, it won't budge.

tekochip
User Rank
Platinum
Re: A software glitch after all
tekochip   11/11/2013 11:36:04 AM
NO RATINGS
That probably wouldn't work once the vehicle is moving, since the brakes would have to combat the engine and the vehicle kinetic energy.  After several seconds the brakes would heat up as well, and the braking capability would be seriously diminished.

RTristani
User Rank
Iron
Re: A software glitch after all
RTristani   11/12/2013 7:21:01 AM
NO RATINGS
One magazine, I think it was Car and Driver, did a test of this.  Even a supercharged Roush Mustang, traveling at freeway speed, with sudden application of brakes and throttle came to a full stop.  Their conclusion was that no ordinary car's engine, even with the car moving at freeway speed, was going to overcome its brakes.

tekochip
User Rank
Platinum
Re: A software glitch after all
tekochip   11/12/2013 9:03:18 AM
NO RATINGS
RTristani  you are correct and that's good information to have!  Here's a link to the article:

 

http://www.caranddriver.com/features/how-to-deal-with-unintended-acceleration

 

Even at 120MPH they were able to bring the car down to 10MPH at full throttle before the brakes started smoking.

Amclaussen
User Rank
Platinum
Re: A software glitch after all
Amclaussen   5/20/2014 2:15:32 PM
NO RATINGS
BUT... That Mustang already has big enough brakes compared to many sedans.

I would not be so sure that a common midsize sedan can be safely stopped everytime in daily driven conditions in a hurry or emergency situation.  And please consider some (many) replacement brake pads and rotors simply do not work as effectively as desired. Many present day replacement rotors simply warp badly in many cases and do not provide an effective braking system. Same for many pads. I'm talking real world reality.  Add less than perfect shop repair results and there is some place for brake overload/failure in this scenario.  A common driver that gets surprised by a lack of braking can easily overheat the brakes and crash.  Another more decided driver (maybe the Roush Mustang one) will be determined enough to properly apply FULL braking on time and result in no crash.  For me it is still an unresolved aspect. Amclaussen.

TR3driver
User Rank
Silver
Re: A software glitch after all
TR3driver   11/11/2013 1:22:23 PM
NO RATINGS
"Does any of this conjecture respond to the notion than the application of full throttle cannot be overcome by the cars ordinary brake system?"

That is a given, under some set of circumstances.  No car has brakes that can dissipate full engine power forever, without becoming overheated and fading to uselessness.  Starting out from stopped, with the brakes cold, most cars will hold, certainly.  But starting out at speed is a lot different.

There is likely a certain element of operator error as well.  For example, when the CHP officer had his throttle stick, he apparently tried to drive the car off the freeway that way, holding the speed down with the brake pedal.  Once the brakes were overheated, it was hopeless.

Rob Spiegel
User Rank
Blogger
Re: A software glitch after all
Rob Spiegel   11/11/2013 1:32:06 PM
NO RATINGS
It's my understanding that Toyota and other car makers have a brake override written into their software so an accelerator malfunction would be broken by brakes.

Charles Murray
User Rank
Blogger
Re: A software glitch after all
Charles Murray   11/11/2013 6:51:58 PM
NO RATINGS
Absolutely true, Rob. But as I'm sure you know, it was not true in 2002.

TJ McDermott
User Rank
Blogger
For the plaintiff ... (but) designated by the court
TJ McDermott   11/8/2013 9:56:56 AM
NO RATINGS
Doesn't that seem contradictory?  Never mind, I'm assuredly not a legal expert.

15 months, 2000 hours, and an 800 page report.  If I was footing the bill, I'd ask for a simulation showing how these discovered safety holes would manifest into loss of throttle control.  Having the simulation run in court, explaining each condition that leads up to loss of throttle control, would seem to be a slam-dunk for the plaintiff.

 

Charles Murray
User Rank
Blogger
Re: For the plaintiff ... (but) designated by the court
Charles Murray   11/8/2013 10:11:01 AM
NO RATINGS
The dynamometer testing was for the purpose of demonstrating that the failsafes did not prevent loss of throttle control, TJ. I don't know if any simulation was run in court. Barr's testimony essentially said this: Skid marks weren't compatible with pedal misapplication; there was no "sticky pedal" recall for this model year; there was no pedal entrapment; and the car had been inspected a dozen times for mechanical problems, such as throttle blockages. Given those facts, along with the fact that dyno testing showed "gaps" in the failsafes, it was more likely than not that a software malfuntion caused the throttle problem.

Greg M. Jung
User Rank
Platinum
Evidence
Greg M. Jung   11/9/2013 9:18:29 PM
NO RATINGS
Very good discussion of a complex issue which will further heighten my awareness to thoroughly test software during design development.

One piece of evidence which swayed me was "skid marks at the accident scene were not compatible with pedal misapplication".  Also, were the plaintiffs able to actually demonstrate this failure on a realy Camry?

Charles Murray
User Rank
Blogger
Re: Evidence
Charles Murray   11/11/2013 6:39:06 PM
NO RATINGS
No, Greg, they were not able to demonstrate the skid mark conclusion on a dynamometer. Measurements made at the accident scene served as the evidence. The 150-foot skid mark was believed to be far too long and the reduction in speed far too small if the driver was not fighting an open throttle, this expert said.

OLD_CURMUDGEON
User Rank
Platinum
MY two cents....
OLD_CURMUDGEON   11/11/2013 9:41:45 AM
As the owner of 3 CAMRY vehicles in the past 20 years, I can categorically state that overall I have never owned a more pleasant all-around vehicle to drive.  And, I've had the full gamut of vehicles from simple, small compacts to large cruisers, and SUVs, to boot.  Each CAMRY has been driven well over 100K miles each with little or no operational problems, and each one has been "treated" to the proper Preventive Maintenance schedules.  I have NOT experienced a single misstep in the throttle control or any other on-board control.

With regard to this "CAMRY" problem, what I don't understand is why it is so peculiar to ONLY CAMRY vehicles.  One would think that TOYOTA would have developed a single Engine Control Module for all it's vehicles, making only minor changes as needed for all their product lines from the COROLLA through their TUNDRA trucks.

With regard to TOYOTA denying or hush-hushing the problem, hoping it to go away by itself, I find that VERY HARD to believe, given the Japanese culture of honesty, integrity.  IF this acceleration problem had occurred en masse to an American vehicle manufacturer, AND it was covered-up with great effort, then I'd say it was typical of the American corporate culture of intentionally deceiving people for the sake of profit, but it's hard for me to swallow that rationale when it comes to TOYOTA, UNLESS this was the result of TOYOTA of America, and NOT of TOYOTA of Japan, where the head honchos reside!!!!

ADIOS!

Ratsky
User Rank
Platinum
Re: MY two cents....
Ratsky   11/11/2013 2:33:00 PM
NO RATINGS
As one who has worked as an engineer for a Japanese company in the auto industry segment (NOT Toyota), and as a (mostly) satified owner of 2 Camries (1992 and 2003): I can readily believe this.  One word can explain the stonewalling; FACE.  This has come to dominate Toyota relatively recently.  They have become extremely defensive, to the point of damaging their formerly sterling reputation for customer service.  Example: my 1992 Camry had a sudden transmission failure after the expiration of the (then) 36K drivetrain warranty.  The dealer replaced the tranny at no cost.  Also, all regular maintenance items were priced well below the competition: example, replacing the timing belt at the recommended 60K interval cost about $180; on a comparable Mitsubishi, it ran $650! On the 2003, the dealers go out of their way to oversell and overservice their cars well beyond factory recommendations (at 3-4x the cost).  I'm still driving that 2003 (~150K miles) but the dealer hasn't seen that car in years because I have an indepoendent garage that is both honest and reasonable.  I had several issues that should have been fixed under warranty but Toyota and the dealer refused to even admit the problems were real.

Regarding parts commonality, Toyota AFAIK to this day has completely independent design teams for each vehicle platform.  They have never bought into the concept of a common HW platform for an ECU type with adaption to the specific vehicle (e.g. different engines, etc.) by parameterized SW.  Much of that is due to the extremely hierarchical nature of Japanese company management; the "old school" folks at the top won't sign off on anything they didn't already know how to do.

OLD_CURMUDGEON
User Rank
Platinum
Re: MY two cents....
OLD_CURMUDGEON   11/11/2013 2:49:36 PM
NO RATINGS
Ratsky:  I totally agree that the present situation in my TOYOTA dealership seems to reflect your experiences.  I've dealt w/ the same service writer for well over 10 years, and he has always been "fair & balanced" (where have I heard that expression before?).  However, he's been on medical hiatus since late last winter, and the other service writers seem to want to suggest that I need to change a lot more items, including the air in the trunk compartment on a far more frequent basis.  Ironically to this post, I will be bringing my CAMRY to the dealer tomorrow or Wednesday for its periodic (90K mile) service.  I can just imagine the litany of items that I MUST attend to immediately!!!!  I have seen my dealership change its M.O. (modus Operandi) considerably since I traded our Windstar over a decade ago on the first CAMRY.  Time will tell.  I hate to say it but we were seriously considering a VENZA, but may now consider the HYUNDAI alternative......  Woe is me!

Ratsky
User Rank
Platinum
Re: MY two cents....
Ratsky   11/11/2013 2:57:24 PM
NO RATINGS
I hate to say it, but my wife's car is a Hyundai Sonata (2006), and our experience with that dealer are much worse!  I've posted before about our adventures with that car's "designed by monkeys" airbag system, and the fact that the "fix" on the recall notice involved removal of the front passenegr seat and shipment to the factory (in California) so they could change out the (not reflashable) seat micro.  Kicker was this did NOT fix the problem.  NHTSA finally forced them to take action beyond a recall: what Hyundai did was send out a glove box label telling the passenger to set the seat fully upright, in the center of fore-aft range; otherwise the airbag would likely be disabled in the event of a crash!  Cross another brand off my list.....

OLD_CURMUDGEON
User Rank
Platinum
Re: MY two cents....
OLD_CURMUDGEON   11/12/2013 7:48:30 AM
NO RATINGS
Ratsky:  Thanks for the heads up.  We have friends close by who have bought & sold HYUNDAIS for the past 10+ years like most people change their socks.  Not exactly sure why, since I never hear any negative comments.  Maybe they just like the "experience" of buying a new vehicle every year or so.

There's a local HYUNDAI dealership in this west central FLA area that inundates the area w/ mail advertising.  There isn't a week that passes that we don't receive at least one or more "SUPER SALE EVENTS of the Century", etc.  Too bad there aren't laws restricting or clamping down on boasting!  This dealership would win the Grand Prize every year.

Funny thing is that the old location of the TOYOTA dealer is almost directly across the highway from the HYUNDAI dealer.  So, you can imagine the competition!!!

To tell you the truth, I've seen a drastic change in the attitude & daily operation of the TOYOTA dealership, and from my perspective, I can't say that I'm pleased with it.  Buying a new vehicle in the coming years is going to be a scary event, I have the feeling.  And, now with the vehicles becoming almost human-like in their intelligence, that scares me even more.....

 

Jim_E
User Rank
Platinum
Totoya Software
Jim_E   11/11/2013 10:29:42 AM
NO RATINGS
Reading one of the earlier articles in Embedded System Design, I was struck by the fact that Toyota supposedly couldn't product the exact code use to create the release version of the ECM software.  Were they just covering up bad code, or is/was their version control that poor?  From what I read there, the NASA guys weren't even given the correct code used in the vehicle.

As a software guy, I too thought that it might come down to the software being the reason behind this unintended acceleration.  Barr's analysis on the failure of the failsafes make me wonder about the reasons behind this.  Was the product rushed to market?  Were hardware engineers writing the software and not grasping some of the software system aspects?  Just sloppy coding by inexperienced programmers?

I sure hope that newer Toyota's have better code, especially since we own a 2013 Sienna....

I was sure to tell my wife that if this situation ever happens, immediately knock the shifter into neutral, and if that doesn't work, turn the ignition key back one click.

Critic
User Rank
Platinum
What has been proven?
Critic   11/11/2013 11:09:59 AM
So what has really been proven?  Toyota losing a civil suit doesn't prove there was anything wrong with the car.  An "expert" saying that the software wasn't perfect doesn't prove it was a cause of the unintended acceleration. 

How does the "embedded expert" know that there was no pedal misapplication?  Did he say this because there was a possibility that the unintended acceleration was caused by a software issue?

The driver did something wrong.  She didn't stop the car!  She should have been on the brakes as soon as she noticed the car was accelerating.  In fact, since she was on the off ramp, she should have been on the brakes, anyway!!!

Brakes, ignition, neutral.  Practice this.  Be able to do it quickly.


No, we don't want more complicated cars.  Keep them simple.

Parris Boyd
User Rank
Gold
Re: What has been proven?
Parris Boyd   11/11/2013 2:06:08 PM
A jury decided that quite a bit has been proven, declaring not only a guilty verdict, but also stating that Toyota acted in "reckless disregard of the rights" of Plaintiffs. Toyota's efforts to cover up pertinent information have come to light, and the attorneys who won the case have not been bashful in exposing Toyota's tactics. Here are a few excerpts from an article:

"...in reporting to NHTSA, Toyota removed the search term "surge" and only used the term "mat," which resulted in only 124 claims being reported to the government agency. This was a deliberate move on Toyota's part and was designed to hide a known defect."

"Internal emails showed that Toyota employees worked extremely hard to 'coach' NHTSA to use Toyota's language when completing unintended acceleration investigations."

"...Toyota withheld certain important software source code from NASA and misrepresented the existence of vital memory protection characteristics of the Camry throttle control system."

Regarding the trial itself, "Toyota could never explain why THE BOOKOUT VEHICLE LEFT A 150-FOOT SKID MARK FROM A LOCKED REAR TIRE PRIOR TO IMPACT (emphasis mine). Mrs. Bookout had first applied the service brakes and then pulled the parking brake, but she couldn't stop the car. Toyota's own litigation testing proved the vehicle should have stopped before its impact with a dirt bank if everything was functioning properly." Furthermore, way back in 2010, Toyota President James Lentz admitted to Congress that floor mats and sticky pedals were not related to 70% of the unintended acceleration claims.

Here's a link to the complete article: http://www.beasleyallen.com/news/toyota-sudden-unintended-acceleration-lawsuit-ends-in-landmark-verdict-2/

Right on for Michael Barr's expert analysis. I applaud Charles Murray and Design News for highlighting the technical aspects of this case.  

 

 

 

Critic
User Rank
Platinum
Re: What has been proven?
Critic   11/18/2013 11:38:01 AM
NO RATINGS
The locked wheel proves that the brake system was at least partially functional.

The 150-foot skid mark proves that the driver had enough time to turn off the ignition and shift to neutral.  Oh, she had enough time to apply the emergency/parking brake?  Apparently she didn't know the proper procedure for stopping the car!!!

Most cars don't stop themselves- this is the driver's responsibility.  Why didn't she push harder on the service brakes???  She was too weak?  Maybe she should not have been driving.  Maybe she had already ridden the brakes, instead of stopping the car.

Maybe there was something wrong with the car, maybe there was not.  We don't know with 100% confidence.  It's wrong to take one side or the other, except in a civil lawsuit, where the burden of proof is only "more likely than not."

Unfortunately, we consumers are going to pay the price for these lawsuits, regardless of what really happened.

Parris Boyd
User Rank
Gold
Re: What has been proven?
Parris Boyd   11/18/2013 12:44:23 PM
"Critic," it's time to stop trying to blame drivers for Toyota's lousy products. Time to face the fact that a jury heard expert testimony and found Toyota guilty by a "preponderance of the evidence," which is the legal terminology for the burden of proof in a civil case. The jury was obviously adamant because it went even further, stating that Toyota showed "reckless disregard" for Plaintiffs rights. Recall King Toyota has left a broad swath of destruction, featuring deaths, injuries, and at least one unjustly imprisoned Toyota driver who was finally relaesed after the facts were exposed. Talk about consumers paying a price...   

AnandY
User Rank
Gold
Re : a software glitch after all
AnandY   11/24/2013 12:13:01 PM
We have to understand that motor vehicle design, especially high end motor vehicle design, is a complex process and its hard to keep tabs on every part of the process. In addition, most of the process is automated and a simple glitch in any could have devastating effects such as the one witnessed in the 'acceleration mishap'. This is a wakeup call to designers not so keen on the minor details.

Charles Murray
User Rank
Blogger
Re: Re : a software glitch after all
Charles Murray   12/11/2013 7:31:02 PM
NO RATINGS
Agreed, AnandY. It's getting tougher for engineers to keep an eye on those "minor details" (especially when there are hundreds of thousands of lines of code), but they have to do it.

Amclaussen
User Rank
Platinum
Was it absolutely necessary?
Amclaussen   5/20/2014 2:52:42 PM
NO RATINGS
Agreed too, Charles.

But it still puzzles me as -Was it absolutely necessary?- (I'm referring to the use or ab-use of electronics in cars nowadays). To me, the use of electronically actuated accelerators is still a case of ab-use. A much simpler and reliable Bowden cable throttle is more than enough to perform reliably. (Unless you consider the phanatic emissions-lowering brigade that pushed that kind of design onto present day cars, obviously. Another fine example of Eco-Illogical design, like lead-free solder, Compact Fluorescent Lightbulbs and other examples where overly 'green' people have caused more harm than good in the end.



Partner Zone
Latest Analysis
Watch as we teardown Amazon's Fire Phone -- the company's first smartphone, designed to compete with iOS, Android, and Windows Phone devices.
Lithium-ion batteries will soon back up the power grid on the Hawaiian island of Kauai, providing the stability to handle intermittent power fluctuations from renewable energy sources.
A relative newcomer to the 3D printing market has developed a 3D printer that can use five different materials in multiple colors for customized creations.
These free camps are designed for children ages 10 to 18. Attendees are introduced to 3D CAD software and shown how 3D printers can make their work a reality. Many classes were nearly 50 percent girls and 50 percent boys.
Take a look through these film and TV robots from 1990 through 1994.
More:Blogs|News
Design News Webinar Series
7/23/2014 11:00 a.m. California / 2:00 p.m. New York
7/17/2014 11:00 a.m. California / 2:00 p.m. New York
6/25/2014 11:00 a.m. California / 2:00 p.m. New York
5/13/2014 10:00 a.m. California / 1:00 p.m. New York / 6:00 p.m. London
Quick Poll
The Continuing Education Center offers engineers an entirely new way to get the education they need to formulate next-generation solutions.
Aug 4 - 8, Introduction to Linux Device Drivers
SEMESTERS: 1  |  2  |  3  |  4  |  5  |  6


Focus on Fundamentals consists of 45-minute on-line classes that cover a host of technologies. You learn without leaving the comfort of your desk. All classes are taught by subject-matter experts and all are archived. So if you can't attend live, attend at your convenience.
Next Class: August 12 - 14
Sponsored by igus
Learn More   |   Login   |   Archived Classes
Twitter Feed
Design News Twitter Feed
Like Us on Facebook

Sponsored Content

Technology Marketplace

Copyright © 2014 UBM Canon, A UBM company, All rights reserved. Privacy Policy | Terms of Service