No-hands harvesting

Harvesting has come a long way since the Pilgrims handpicked corn to celebrate the first Thanksgiving.

Consider the switch from horses to engines, the debut of new types of equipment like McCormick's reaper, and genetic developments such as hybrid seeds that resist disease. The next big advance: robotic harvesting.

At the Robotics Engineering Consortium (REC) in Pittsburgh, PA, researchers from Carnegie Mellon University and New Holland, which makes farming equipment, have such a project under development: Demeter.

The windrower can cut crops like alfalfa auto-nomously using a machine vision system--or the GPS (global positioning satellite) system, when there's no crop line to track. It's based on technology that Carnegie Mellon robotics researchers developed for legged planetary rovers like the robot Dante II, which explored the crater floor of a live volcano in Alaska two summers ago.

"Our strategy is to use what we know in terms of navigation, perception, and tracking and apply it to harvesting," says Project Leader Kerien Fitzpatrick. "Demeter is a prototype for other large, mobile machines operating outdoors that do heavy-duty jobs like excavating, mining, and timbering."

The project was conceived by mobile-robotics pioneer Red Whittaker and Fitzpatrick in 1993. "The harvester may be one of those transformation machines that changes the world," says Whittaker. "This is definitely an instance of a robotics technology reaching into an industry and jolting it into the next century."

When REC researchers approached New Holland to join them in the project, the firm's officials quickly agreed. "Sometimes in order to make things happen you have to have some faith," says New Holland Chief Engineer Walter V. Pype. "When these people talked to us you could see in their eyes they were sincere and really wanted to do something revolutionary."

New Holland's research expertise lies in mechanizing farming tasks, such as cutting and packing hay. "This is a new experience for us," continues Pype. "The REC people were used to working in space. We always say how harsh space is, but earth-bound reality is probably harsher--what with dust, debris, people, dogs, rocks, and holes in the ground."

Both sides are learning from the venture. New Holland found out that computer control lets it add such features as driver-adjusted control sensitivity and reverse steering in hours. The robotics researchers now understand that saving a length of cable or a couple of fasteners really does matter when you're designing for manufacture--as opposed to designing one multimillion dollar machine to explore a planet.

How it works. Cutting alfalfa isn't as easy as cutting your lawn. Farmers find that keeping the cutting head lined up so that it doesn't trace over already cut crop or leave crop standing is one of the most strenuous parts of the job.

Demeter operates on two technologies, which the researchers split so that they could be modularized and used in other mobile robotic systems. One, FieldNav, combines hardware and a physical control layer of software that automates the machine. It locates the harvester's position in the field, ensures safe operation, and accurately times and positions tag events. The second, FieldHand, is a behavior-based software architecture that performs such high-level tasks as planning the harvest of fields and interacting with a human supervisor.

Before the harvester could respond to the software, the researchers had to replace mechanical linkages with electronic connections. They also had to add computers, sensors, and safety circuits.

The electronics that comprise FieldNav include: inclinometers; a compass; wheel encoders; a differential GPS receiver; a Motorola 68HC11 microcontroller and 68040 motherboard with expansion modules; and a custom interface board consisting of hydrostatic-pump signal amplifiers, signal-conditioning hardware, sensor-monitoring firmware, and watchdog relays for operator and equipment safety.

Robotics components aren't yet off-the-shelf commodities. So Fitzpatrick, Demeter coworker Henning Pangels (REC assistant software director), Chris Fromme, and Bryon Smith formed Sextant Systems in Pittsburgh. Sextant designed FieldModule--the hardware and connections needed for FieldNav. Currently, the system resides in a standard VME enclosure, but the firm is working on shrinking it to the size of a beer can.

"We want to have the components ready," says Fitzpatrick, "so that if we want to automate a combine, which is a more complex machine, the development time would be extremely short."

Also on board Demeter is a Sun SPARCstation and two Sony video cameras--one on either side of the cab. The cameras send input to the computer, which runs a program--part of FieldHand--to detect the subtle color and texture differences between cut and uncut crop. These differences let the machine determine the "cutline," which it uses to position its cutting head for the next row.

A trailer houses the base station, which includes a Sun UltraSparc workstation connected to a wireless radio Ethernet bridge for such high-bandwidth telemetry as video. The base station lets an operator monitor and control the harvester. Also at the base station is a separate RF modem that handles only three functions: start up, stop, and emergency stop.

Computer-aided systems let operators run the harvester at the maximum cutting speed of 6 mph. Humans alone inevitably slow down to average 3 or 4 mph over the course of a day.

Demeter's generic hardware can be configured to do most anything, but it's really overkill. Now that the team has finetuned the vision algorithms and knows explicitly what the vision system needs to do, the next step is to simplify the hardware. "Our plan now is to put a DSP in the camera to reduce cost and system complexity," says Fitzpatrick. "If that works, we could do the vision algorithms all in hardware--you don't even need code."

Roll out. "Harvesting an entire field without a driver on board--that's the ultimate dream," says Pangels. However, the team realizes there will be more of a staged introduction of the technology.

"You can't just drop something in front of farmers that's so new and so different and expect them to trust it to get their harvest in," Pangels notes.

The first commercial implementation will be an "intelligent" cruise-control option called Fieldcruise. An operator would sit at the controls, but could press a button to let the system drive itself once it correctly determined the cut line.

Next could come an unmanned drone harvester that operates in tandem with a manned version. (Windrowers usually work in groups of four or five when harvesting a field.)

Fitzpatrick envisions a future when teams of unmanned robot harvesters bring in crops, managed only by a farmer with a portable computer over cellular phone lines.

For New Holland questions remain unanswered: "How simple will it be for the farmer to operate?" asks Pype. "If he has to be a computer wizard, forget it--he'll hop on the machine and do it himself." The firm also remains unclear about how much the technology will add to the cost of the machine. Not all the hardware is yet defined, and the system continues to get less complex as time goes by.

Farmers who have seen the harvester with nobody inside are enthusiastic, says a surprised Whittaker--who says he has consistently underestimated the impact of robotics.

"Some robots have changed the way we do search and recovery undersea--we no longer send people underwater in deep-diving suits," he notes. "or the way we do nuclear accident response. This notion of mobile, unmanned working machines that operate for our benefit and don't require human attention is in itself revolutionary."

Cat excavator joins robotic ranks

An REC team comprising engineers from Caterpillar, Peoria, IL, and Carnegie Mellon's Robotics Institute is working to automate Cat's Model 325 hydraulic excavator. The goal: improved productivity and reduced mining and earthmoving costs.

In a typical mass excavation application, a loading machine digs material from a face and dumps it in a truck, with a throughput of hundreds of trucks per day. The repetitive process continues day and night and in most weather conditions.

"This is a perfect application for automation because the work is both demanding and boring," says John Bares, principal investigator for the Autoload project. Automated excavators could run 24 hours a day without demanding coffee breaks. Companies would save money from not having to pay three shifts of workers and having the machines run continuously.

Cat's excavators must meet many challenges from operating in harsh environments to precisely controlling hundreds of horsepower and moving tons of material in just seconds. Methods employed to meet these challenges must do so reliably and cost effectively. "The REC work is not just a laboratory study," notes Ken Bennett, director of research and development at Caterpillar, "but is the first step in the realistic application of robotic technology to earthmoving."

"Many robotic systems over the last 10 years were developed to remove humans from hazard," adds Bares. For example, the robots Red Whittaker and company designed to clean up Three Mile Island and collect gas samples from volcanoes saved humans from entering hazardous environments.

"Now we're starting to see an evolution of the basic technology into applications where productivity and reduced cost--not safety--are the bottom line." Once robotics is actually saving companies money, expect corporate resistance to the technology to magically disappear.

Timeline for Design

Late 1993-- Red Whittaker and Kerien Fitzpatrick conceive idea for a roboticized harvester, Demeter.

April 1995-- Development starts.

September 1995-- Demeter successfully cuts alfalfa in a Pennsylvania field.

August to September 1996-- Demeter tested in the circular fields of Garden City, KS.

October to November 1996-- Demeter tested in the large rectangular fields of El Centro, CA.