There were many new things I had to learn when I was a young engineer just out of college. I began my career at a high-tech defense firm. We received formal training on our products (mostly airborne radars for fighters), but there were other things to learn on the job. The military aircraft use 115V, 400Hz AC power instead of our terrestrial 60Hz flavor, and 28 volts for DC power and signals. It was easy to understand the reason for 400Hz power -- transformers that would have been big and very heavy -- at 60Hz were much smaller and lighter at the higher frequency.
Within a few months, I was in charge of all the hardware in a radar software development lab. I had all kinds of challenging projects thrown at me -- developing a microprocessor-based console interface to connect the ancient radar CPU to the more modern SW development tools, integrating the radar and some of the test equipment with other new equipment, and supporting the testing of new software. It was both challenging and fun.
One day a problem arose that kept interfering with our work. Every day after lunch the radar and the test bench would shut down. The power to the radar and to the 400Hz equipment in the test set would drop out and whatever work we were in the middle of would be lost. If we restarted things right away, it would often happen again, right away. But at other times of the day, we could run for hours with no problems. Someone dug up a power line monitor that would record out-of-tolerance voltages on the 60Hz and 400Hz feeds, but it never showed us any voltage glitches. Yet the shutdowns continued.
About the time the SW engineers were getting testy about the problem, a few other labs were reporting similar problems. They wanted to blame it on us: "Aren't you the guys who have the power problems?" they'd ask. "Now you're messing us up too!" "No, it wasn't us," I assured them, but it was time to get serious about tracking the problem down.
First we found the schematics for the test bench which included circuit breakers, power contactors, and controls for the radar set and its associate equipment, including the high-powered 400Hz cooling fan that put lots of cooling air through the radar and sounded like a jet engine when it started up. There were a few interesting clues, but no obvious source of our problem there.
We noticed that on the test bench, as it is in the actual aircraft, the radar provides output signals that power the main contactors, so that it could safely shut itself down in the case of a power supply failure. But this was a mature product and there was no problems with these signals in the aircraft or at lower level tests. Besides, this test bench was many years old and the problem was a new one, so it didn't seem to be a design problem.
Next, we went to talk to the facilities guys to learn about our power sources. It was there that things began to fall into place. Talking over the roar of several 60Hz to 400Hz rotary motor generator sets which were producing our 400Hz power, the facilities engineer mentioned that the brushes on the generators had recently begun to wear out much more quickly than usual. Then we looked at the building wiring and saw that my little lab was at the extreme far end of a long transmission line that snaked its way through our factory, then up to the radar labs on the roof of the building. Several long branches divided off to the "shake and bake" reliability labs and to the factory test sets on the main floor of the building.