The Adventure of the Malfunctioning Modem
An engineer discovers that it’s not always a bug in the software or hardware that is causing all the trouble. And that troubleshooting can be a really grungy job.
By Dennis Coburn, Contributing Writer
Back when modems were gray plastic boxes that worked at the screaming rate of 300 baud, I was involved in the design and installation of an environmental monitoring system for the US Army Corp of Engineering in Florida.
The system was made up of a number of remote monitoring stations housed in fiberglass “coffins,”each consisting of a sensor package, a data recording device, and a telemetry link. These packages were placed along a line roughly centered from East to West and running North to South along the length of Florida. The telemetry link included a 300 baud modem at each site connected to standard dial-up phone equipment. Each station was interrogated once a day and transmitted its stored data to a base station for processing.
The system was solid in its operation–not having to cope with low temperatures (the recording device was an analog tape recorder converted for digital operation) and featuring its own battery-backed power supply running off of the local mains.
However, after a number of months in operation a problem arose that intermittently prevented the interrogation of some of the remote monitoring stations.
By this time I had moved on from the company that was responsible for creation and installation of the system. Since no one was left at the company who knew much about the system and since I had worked closely with the Army COE personnel during the development, I was called upon to analyze and repair the problem.
Conveniently, I had just recently made a decision to quit my job and start my own consulting business. The Army COE contract was just what I needed.
I packed up and left New Hampshire for Florida. Upon arriving at the first site of three that were malfunctioning. I discovered that the site was fully functional, and I found no way to make it fail. The second site proved to be the same.
Whether it was the time of day or just luck, when I reached the third site I found it was still malfunctioning, much to my relief. Every engineer knows that it’s always easier to fix something that’s “broke.” However, the only potential problem source that I could identify seemed to be within the modem. So I tore into it, though I wasn’t really expecting to be able to repair it on site since it was provided by the phone company and I had no information on it at all. Still, I had to act as if I was doing something so I had at it.
To my surprise and shock as I pulled the plastic cover off, I had an unpleasant surprise: Half a dozen 2.5-inch-long roaches scurried out of the box!
It was then that I discovered both the cause of the problem and why it was intermittent.
It turns out that roach feces, when fresh, at least, shorted out some of the modem’s circuitry. Apparently, when dried, depending upon the exact location on which it was deposited, the problem went away.
The solution turned out to be pretty simple. I scrubbed the printed circuit board clean, dried it and reinstalled the cover. Then I taped up the entry way for wiring that the pests had used to enter their favorite toilet.
Lesson learned: It’s not always a “bug” in the software or hardware that causes trouble. Sometimes it’s what the bugs leave behind.
JRD commented:
A different kind of pesky critter, and a different kind of field. This phenomenon was a topic of humorous conversation in the Bell Labs Undersea Light community in the mid-eighties.
------------------------------------------------
From the Wall Street Journal, 26-June-86
By Bob Davis
Staff Reporter of The Wall Street Journal
Just when American Telephone & Telegraph Co. thought it was safe to go into the water, sharks began dining on its newest undersea
telephone-communications cable.
It seems the sharks just can't get enough of AT&T experimental underwater fiber-optic telephone cable near the Canary Islands. They
munch on its plastic covering, gnaw on its electrical innards and eventually short-circuit it-even though they may electrocute themselves in the process. At least, "we came up with some pretty effective shark bait," says an AT&T spokeswoman.
At first, AT&T engineers didn't know what was causing the cable failures. Then they raised the cable and found rows of shark teeth
sticking out of it. "Sharks will always be attracted to magnetic fields," which the fiber-optic cables create, says James Barrett, an
AT&T engineering official.
------------------------------------------------
The species of shark was identified from the teeth, and the marine biologist who made the ID wrote a article about the incident for Natural History Magazine. The sharks extremely high bite pressure made armor impractical. To the best of my recollection, lower currents and better shielding solved the problem.
Joe commented:
One year at the onset of cold weather my furnace wouldn't work. Being an electronics tech, I checked out the thermostat, voltages, wiring etc; everything seemed to work properly but the gas. I called the gas company and they explained that their pressure was only 0.25 psi. I tore into the gas hardware and discovered that a spider web was plugging up the nozzle.
DaveD commented:
Long ago retired from Bell System. Roaches were a persistent problem in Southern states operating areas especially in telephone sets. Since it was impossible to plug-up all of the holes (roach entrances) in the instrument We used a product called "Insectape" which was a tape square impregnated with some sort of repellant stuck to the inside of the baseplate. It worked wonderfully, wish I could recall who made it. I think that we originally located the product through HUD.
JWF commented:
I worked for a company that made an electronic pump controller for domestic and agricultural use. The enclosure was vented so condensation could evaporate. We started to have epidemic failures in one particular part of the Australian outback and after some hoohaa found a tiny ant (almost invisible to the naked eye) was getting through the fine mesh protecting the vent and nesting in the electronics with the inevitable result. It turned out to be a species unknown to science! A really REALLY fine mesh solved the problem.
RS commented:
Had similar problem in NYC with rats, those thing eat everything.
Jon commented:
I lived near the company where I used to work, so I was the call-in guy when the alrm went off. One night, we had three false alarms on one of the motion sensors for no apparent reason. Finally I happened to notice a shadow in the motion sensor. A moth had moved in, and seemed to have an amazing ability to move in front of the sensor each time I fell asleep!
jed martin commented:
i've seen where mice make nests in subwoofer cabinets. still haven't figured out a "humane" way to keep them out. steel wool would detune the box. screening over the port would hiss. the only way to keep them out is to put strips of adhesive backed copper foil in the port opening connected to the subwoofer's power supply.
LB commented:
I have been to AM bvroadcastantenna tuning units in the south of the US where fire ants had invaded. With several kilowatts of RF, the RF loving ants quickly fried and piled up in foot high drifts!
Tom R commented:
The only thing we found that would keep mice out of the equipment was packing every small opening with steel wool.Those critters will eat through just about anything else.
William Ketel commented:
I had the roach problem with a control box in Metairie Louisiana. MY evaluation, based on how well the box was sealed, is that roaches can spontaniously generate out of air. Boxes of mothballs and bug killer were the solution.
steve commented:
From the wikipedia page for Grace_Hopper (no link cause URLs are disallowed in comments).
"While she was working on a Mark II Computer at Harvard University in 1947, her associates discovered a moth stuck in a relay and thereby impeding operation, whereupon she remarked that they were "debugging" the system. Though the term computer bug cannot be definitively attributed to Admiral Hopper, she did bring the term into popularity.[12] The remains of the moth can be found in the group's log book at the Smithsonian Institution's National Museum of American History in Washington, D.C.[13]"
The wikipedia page also includes a photo of the log page.
Larry commented:
Back in the day of full height 5-1/4 floppy drives one brand had a door that when left open was large enough to let a mouse to get in and eat the very tasty wires to the R/W head.
dduston commented:
I have seen air conditioner contact relays fail because of sugar ants that love the grease around contacts. They get squashed when the contactor engages, causing corrosion of the contacts. I report the failure as too much "antacid" applied to the contactor.
Nel commented:
In our customer repair dept., it was a ritual to "fume" and clean all Florida units (and some other warm humid states) due to roaches. Most failures were due to roaches and their droppings. Roaches like warm electronic enclosures.
Larry Taft PE commented:
Houston, TX: Fire ants build nests in the outside electrical outlet boxes and occasionally cause a blowout of the outlet.
Peter commented:
I have seen spider webs in lighting control centers, but all that did was kill the spiders, so ne problem there. However mice and ground hogs have the strongest teeth in the universe and can chew thru most electrical cables.
And then there was the deteriorated concrete light poles. Only 2 things will hurt this pole, a vehicle hit, or dog urine will make it crack.
ran commented:
That's what conformal coating is supposed to prevent
Catraeus commented:
I had heard at one point that the reason we call the process debugging was that those very same roaches' cousins would enjoy the warmth of the vacuum tubes used to make the gates of early computers. They would get between battery and grid then conveniently die. The circuits needed to be debugged to get them working again. Is that Hardware or Software?
RM Cox commented:
I have seen snail tracks eat the soldermask, solder and copper from circuit boards.
JT Klopcic commented:
We had the same problem, except with mice. Mouse urine is highly corrosive on printed circuit boards, so the outcome was not as favorable, however.
Sponsored Content
Design News Partner Zones
Light Matters: The Unsung Heroes of
Modern Health Care
First, let's define "no-compromise." In an ideal configuration, this lamp would use a high-brightness LED (HBLED) that is built into a small, integrated package and is able to produce a large quantity of focused light, operate with a high level of reliability and generate no audible noise. Is this difficult? Yes, but it is possible.
Read More
Design Engineers' Portal for Sensing and Machine Safety
Whatever industry you're in, or whatever product you manufacture, the right sensors to automate your plant, and to improve your overall efficiency, quality and safety are a must. You'll find Banner Engineering to be an amazing resource of products, training and people with expertise.
Test & Measurement World Machine Vision & Inspection Report
Topics include machine-vision software, Power over Camera Link, thermal imaging and frame grabbers. Read More















