HOME  |  NEWS  |  BLOGS  |  MESSAGES  |  FEATURES  |  VIDEOS  |  WEBINARS  |  INDUSTRIES  |  FOCUS ON FUNDAMENTALS
  |  REGISTER  |  LOGIN  |  HELP
Comments
View Comments: Newest First|Oldest First|Threaded View
<<  <  Page 2/2
phgphd
User Rank
Iron
Re: Big mistake
phgphd   9/16/2013 3:15:36 PM
NO RATINGS
Your point is well taken.  Thank you for your honesty.

John E
User Rank
Silver
When to do complete regressions
John E   9/16/2013 11:07:34 AM
NO RATINGS
I make this decision on almost every project my team works on.  Our policy is to do a full test pass on the final FW and HW revisions before we release a product.  The reality is that we are often pressed for time and must make a risk assesment on the "small change" that the FW team did at the last minute.  We often choose to do a focused regression pass, and not the full test sweep; but we do it knowing we are taking a risk, and make sure that the project team agrees.  We build industrial products, not medical or military, so the risk is more contained.

BrainiacV
User Rank
Platinum
Re: Big mistake
BrainiacV   9/16/2013 11:03:37 AM
NO RATINGS
I agree with you, the best advise I've ever gotten has been people telling me their horror stories.  Many a time when stuck on a problem, I'd reflect on some similar problem I had been told and it would lead to a solution.

One time, just before I left for a customer's site across the country to install some software I had written, my manager had been telling me of an instance he had found the hardware engineer had addressed all the interface cards to the same location.

Later on the job site, my program was working, but every once in a while the system would mysteriously crash.  I had to develop the software without the benefit of having a computer that ran the OS, nor did I have access to the hardware itself.  Since I only had the spec sheets, I wrote the program very conservatively and since it was interrupt driven, I didn't know how long between receiving and processing the interrupt, that the background process would get control to deal with the data.  So I placed everything in ring buffers.

After the program crashed again, I checked the buffers and found status values that should not have been there.  Remembering the problem my manager had, I chased down the hardware engineer and had him check the interface board addresses.  They had several modems in addition to the serial interface I was talking to.  It turned out one of the modems had the same address.  When I'd query the hardware, more often than not, a modem would respond instead of the serial interface.

What was crashing the system was that eventually an interrupt would be missed and the system would hang waiting for a return that would never come.

I doubt that it would occured to me to have the addresses checked if my manager hadn't just happened to mention a similar problem he had solved.

Battar
User Rank
Platinum
Bad missile
Battar   9/16/2013 9:14:37 AM
NO RATINGS
Friend of mine worked on missile development somewhere in the world. When they did a small navigational software update they re-ran all the tests with the missile attached to the aircraft wing - on the ground. When a customer fired one of these missiles in a war zone (cant go into details) it went ballistic, hit something it shouldn't have, and caused a huge diplomatic fuss. Investigation showed that with a strong crosswind, the navigation loop didn't complete before the it was interrupted by the next sensor update, and the missile effectively got lost. An air test would probably have shown up the fault. Moral is , as NASA always say, test as you fly, and fly as you test. 

vimalkumarp
User Rank
Gold
regression test
vimalkumarp   9/15/2013 2:16:51 AM
NO RATINGS
 in a mission critical, life critical application regression test must never be bypassed.

armorris
User Rank
Platinum
Big mistake
armorris   9/13/2013 2:17:29 PM
I don't know why I didn't send the technician back to test the entire circuit card. I guess I just thought that the simple timer circuit could not have affected other circuits. I got really chewed out for that mistake. Needless to say, I never made that mistake again. That was fairly early on in my career and the first time I had ever redesigned a circuit in a piece of flight hardware.

I'm sure that there are plenty of engineers who have made mistakes that they will never tell anybody about. Telling this story can't affect my career as I'm retired, but it might prevent some young engineer from making the same mistake.

TJ McDermott
User Rank
Blogger
Re: Suprised
TJ McDermott   9/13/2013 11:46:48 AM
NO RATINGS
Regression testing is necessary for everything.  Rockwell Automation routinely delays firmware releases because of regression test failures.  One can grumble about the delay (and I do), but I am also very appreciative that Rockwell DOES do exhaustive regression testing.

tekochip
User Rank
Platinum
Semiconductors Too
tekochip   9/13/2013 10:49:17 AM
NO RATINGS
When I was with a  semiconductor company we used to see similar issues as the components went through a die revision.  the new part would meet all the specifications of the old part, but somewhere done the line there was a circuit using an unspecified feature of the old part and suddenly their circuit wouldn't work any longer. 
 
The worst case example is a customer I had that used the part above the absolute specified supply rating.  He tested the parts and burned them in and was happy accepting a small failure rate because it saved him buying a regulator on his board.  A new die revision no longer allowed him to be as frugal.


naperlou
User Rank
Blogger
Suprised
naperlou   9/13/2013 9:11:37 AM
NO RATINGS
Andrew, I am really suprised at the testing done on your change.  I worked on spacecraft and other military systems.  Any change would require a complete regression test, especially in a weapn system or man critical system, in my experience.

<<  <  Page 2/2


Partner Zone
Latest Analysis
The Window Watcher stops the burglar before he does damage or enters the house. House alarm service companies set off alarms and call the service only after the burglar has damaged and entered the house.
If you’re designing a handheld device or industrial machine that will employ a user interface, then you’ll want to check out the upcoming Design News Continuing Education Center course, "Engineering Principles Behind Advanced User Interface Technologies.”
Brooke Williams of Texas Instruments explains how TI’s new TDA3x chip will help future vehicles “see” all around themselves.
It's been two years since the Mac Mini's last appearance on iFixit's teardown table, but a newly revised version joins Apple's lineup this week.
More often than not, with the purchase of a sports car comes the sacrifice of any sort of utility. In other words, you can forget about a large trunk, extra seats for the kids, and more importantly driving in snowy (or inclement) weather. But what if there was a vehicle that offered the best of both worlds; great handling and practicality?
More:Blogs|News
Design News Webinar Series
10/7/2014 8:00 a.m. California / 11:00 a.m. New York
9/25/2014 11:00 a.m. California / 2:00 p.m. New York
9/10/2014 11:00 a.m. California / 2:00 p.m. New York
7/23/2014 11:00 a.m. California / 2:00 p.m. New York
Quick Poll
The Continuing Education Center offers engineers an entirely new way to get the education they need to formulate next-generation solutions.
Oct 20 - 24, How to Design & Build an Embedded Web Server: An Embedded TCP/IP Tutorial
SEMESTERS: 1  |  2  |  3  |  4  |  5  |  6


Focus on Fundamentals consists of 45-minute on-line classes that cover a host of technologies. You learn without leaving the comfort of your desk. All classes are taught by subject-matter experts and all are archived. So if you can't attend live, attend at your convenience.
Next Class: 10/28-10/30 11:00 AM
Sponsored by Stratasys
Next Class: 10/28-10/30 2:00 PM
Sponsored by Gates Corporation
Next Class: 11/11-11/13 2:00 PM
Sponsored by Littelfuse
Learn More   |   Login   |   Archived Classes
Twitter Feed
Design News Twitter Feed
Like Us on Facebook

Sponsored Content

Technology Marketplace

Copyright © 2014 UBM Canon, A UBM company, All rights reserved. Privacy Policy | Terms of Service