An engineer discovers that it’s not always a bug in the software or hardware that is causing all the trouble. And that troubleshooting can be a really grungy job.
By Dennis Coburn, Contributing Writer
Back when modems were gray plastic boxes that worked at the screaming rate of 300 baud, I was involved in the design and installation of an environmental monitoring system for the US Army Corp of Engineering in Florida.
The system was made up of a number of remote monitoring stations housed in fiberglass “coffins,”each consisting of a sensor package, a data recording device, and a telemetry link. These packages were placed along a line roughly centered from East to West and running North to South along the length of Florida. The telemetry link included a 300 baud modem at each site connected to standard dial-up phone equipment. Each station was interrogated once a day and transmitted its stored data to a base station for processing.
The system was solid in its operation–not having to cope with low temperatures (the recording device was an analog tape recorder converted for digital operation) and featuring its own battery-backed power supply running off of the local mains.
However, after a number of months in operation a problem arose that intermittently prevented the interrogation of some of the remote monitoring stations.
By this time I had moved on from the company that was responsible for creation and installation of the system. Since no one was left at the company who knew much about the system and since I had worked closely with the Army COE personnel during the development, I was called upon to analyze and repair the problem.
Conveniently, I had just recently made a decision to quit my job and start my own consulting business. The Army COE contract was just what I needed.
I packed up and left New Hampshire for Florida. Upon arriving at the first site of three that were malfunctioning. I discovered that the site was fully functional, and I found no way to make it fail. The second site proved to be the same.
Whether it was the time of day or just luck, when I reached the third site I found it was still malfunctioning, much to my relief. Every engineer knows that it’s always easier to fix something that’s “broke.” However, the only potential problem source that I could identify seemed to be within the modem. So I tore into it, though I wasn’t really expecting to be able to repair it on site since it was provided by the phone company and I had no information on it at all. Still, I had to act as if I was doing something so I had at it.
To my surprise and shock as I pulled the plastic cover off, I had an unpleasant surprise: Half a dozen 2.5-inch-long roaches scurried out of the box!
It was then that I discovered both the cause of the problem and why it was intermittent.
It turns out that roach feces, when fresh, at least, shorted out some of the modem’s circuitry. Apparently, when dried, depending upon the exact location on which it was deposited, the problem went away.
The solution turned out to be pretty simple. I scrubbed the printed circuit board clean, dried it and reinstalled the cover. Then I taped up the entry way for wiring that the pests had used to enter their favorite toilet.
Lesson learned: It’s not always a “bug” in the software or hardware that causes trouble. Sometimes it’s what the bugs leave behind.