Why Did My Test Fail?

If you have designed anything, you have likely failed a test at some point. We have so many requirements to meet, and there are so many analysis assumptions involved. For new products and new technologies, it is unrealistic to expect to fly through every test the first time.

As an example, I once taught a Senior Design class at a local college. Senior engineering students with electrical and mechanical degrees in the balance combined to work on industry-sponsored projects. Over the 3 years I was involved, every one of the 30 projects failed the first full-scale test. You can imagine the panic when they thought they would not graduate! Their life was ruined!

One aspect of my job was to notice this point in time and calm the student team down. It was not about the failure, but instead how they would respond to it. This is far easier said than done. The typical student reaction was to guess at a new design and test as soon as possible. There was a panic to get back to testing.

Is this really any different than in normal industry situations? Management does not want to hear about test failures. They want a simple and confident path forward. In my opinion, we tend to sugar-coat any failures but we have the same panic as the engineering students. Professionals simply hide it better.

Related:Strategic Testing During Development

To be honest, I dislike when people talk about failure. I know it happens. But failure is often a major source of learning. Someone once stated, “You are only a failure if you think you are a failure.” Industry tends to punish failure. If we change our view, we should reward learning. Plus, if we are not failing at some point, are we really making any progress? Failing and learning can be painful. I once heard from a Fortune 500 CEO that one of his secrets to success is acceptance of mistakes. We really need to do better with this. Heck, we should even WANT failure. Sorry—got on a tangent!

What can we do when the test fails? First, we need to understand the specifics of the failure. There are many potential sources of failure. We also could review the test setup. Maybe our measurement system was not appropriate? It is possible our design is good, even if the test failed.

Was the design built right? A defect in construction could be our resolution. Was the analysis correct? Maybe there was a bad assumption or boundary condition. Any analysis is a simplification, so maybe we need to increase the resolution to dial in our design parameters. Maybe our component, subscale, or bench testing was not representative. Additional testing might be needed to expose the true performance.

Related:Best Online Graduate Programs for EEs

Whatever the source, it is useful to understand what went wrong and determine how to move forward. Guessing at an answer and passing a test still means we do not understand what is going on. The chance for future failures in manufacturing or in the field is high when understanding is lacking. Our experiences with Design of Experiments show it is a great tool to understand a system early in R&D. It can empirically explain our measurement system, our test configuration, the manufacturing process, or the design parameters. This provides some understanding and can point to the science behind our solution. For years, I have encouraged my team and my students to fail early and often. Fail faster. I believe I first heard this from Tom Watson of IBM. Execute the critical tests first, not at the end of development.

Regardless of the method used, we want a confident path forward to avoid repeated test failures. More importantly, we want to avoid downstream failures after product launch. The students were visibly shaken when their test failed.

Do we really respond better than the students, or do we have similar knee-jerk reactions to test failures? Do we better understand our design or process? Do we understand the science? Or do we quickly make changes and cross our fingers that we pass the test next time? Curious to hear what you have experienced over your design life. Please share with us and maybe we can include your response in a future column!

Related:Designing R&D Facilities that Work: From the Inside Out