By Jean Donato, Contributing Writer
Putting an engineer on surveillance detail solved this perplexing case.
Back in the 1990s, I was a systems integrator for a major control systems manufacturer. These control systems used a combination of sophisticated in-house software, as well as custom and off-the-shelf hardware. The latter mostly included Unix servers and workstations.
The integration and testing was typically performed at the company premises. We had access to a large staging area, where a few multi-month (or multi-year) projects were being handled in parallel. Each project made use of dozens of these computers and workstations, manufactured by a well-known but now-defunct company.
Since many of these projects were very similar, this was routine work which usually yielded few surprises - except during one hot summer. In the course of staging one of these projects, the integrator found that most of its computers had unexpectedly rebooted overnight. Operating system error logs revealed nothing useful, and no other software would have been running on these computers during their reboot. The next morning, the problem was seen again. And so on, nearly every morning after that!
Suspecting a sinister motherboard or power supply hardware problem, the computer manufacturer was contacted. They swapped parts and more parts. Still, the reboots persisted. The AC power quality was monitored and no anomaly was found. The operating system was also updated and patched, but to no avail.
One fact was clear: Suspiciously, the reboots always happened around the same time every night.
Finally, the decision was made to park an engineer on the test site overnight, with the simple instruction to keep his eyes peeled. Dimly lit and very quiet, the staging area was devoid of any other humans. Our engineer must have thought this red-eye exercise was a major time waster. Then in the middle of the night, he saw the security guard, a recent hire, making his rounds. In doing so, he dutifully walked into every staging area, including the one in which the rebooting computers were located. As he walked in front of the row of workstations and servers, the powerful radio on his belt was a mere few inches away from their keyboards. Our engineer could not believe his eyes: As the guard moved in front of this row of keyboards, nearly every computer behind him rebooted instantly and silently.
The next day, it was confirmed: Simply waving the guard’s radio a few inches from one of these keyboards generally caused the connected computer to reboot instantly. The keyboard was branded and supplied by the same computer manufacturer.
Upon learning of this, the manufacturer theorized that the computers were affected by the walkie-talkie’s strong signal coming in through the keyboards, so they improved the shielding inside the plastic housing of the keyboard.
As an initial test, they applied a metalized shielding spray to the underside of the keyboard’s plastic shell. After it dried and was reconnected to the computer, it was verified that the system became immune to the guard’s powerful walkie-talkie, as it should have been in the first place.
Following additional and more rigorous testing, a new official revision of the keyboard was produced and this became the minimum revision that we could ship to our customers.
The case of the rebooting computers had thus been solved.
Jean Donato graduated in 1990 in Electrical Engineering, specialized in Computers, in Montreal, Canada. He has been working for almost 20 years in various industries, with positions related to project management, control systems engineering, worldwide customer support and information systems.