The great forest fires consuming the western half of the U.S. have spawned interest in the use of drone technology to help fight and even prevent the outbreak of such fires. Currently, drones with traditional cameras are used to collect vital information about ongoing fires. Armed with this real-time information, firefighters learn about imminent dangers to help them focus their efforts where they can do the most good.
Are there other drone-enabled technologies that might be applied to fighting forest fires? One possibility is hyperspectral technology, which provides images in greater detail than traditional-visible-spectrum Red-Green-Blue (RGB) camera systems. This additional detail permits the human viewer or machine learning (ML) system to “see” more details about the image, i.e., the materials that make up the image. Current implementation areas for hyperspectral imaging include the medical, optical sorting, remote sensing, and even agricultural markets.
There are other perhaps more esoteric applications where hyperspectral (HS) imaging is proving it’s worth – such as in examining the paintings of Leonardo Da Vinci. Imec, a major researcher in nanoelectronic technology, have used their HS single-chip imagers to see the different chemical composition in Da Vinci’s Laster Supper masterpiece to determine the oldest layers of the painting. The capability of HS cameras to divide the light reflected by an object into many narrow spectral bands means that these cameras can capture a spectral signature for each pixel in an image.
Combining such detailed information with supporting images from traditional camera enables them to study the fine techniques of painting and derive characteristics such as the way in which Da Vinci held his brush. This information can provide a form of digital signature for the artist.
Such an approach is currently being used by an American art historian Jean-Pierre Isbouts to determine if Da Vinci had a hand in the painting of a 16th-century copy on canvas of Leonardo's Last Supper in the abbey of Tongerlo in Belgium.
But how can this same technology be taken from the environment of a pristine museum to be used to predict and monitor raging forest fires? To find out, Design News (DN) contacted Wouter Charle, the team leader of the hyperspectral imaging (HIS) software at imec. What follows is a portion of that discussion.
DN: How can hyperspectral technology be used to determine where forest fires might start or the path of a fire once it has started?
Wouter Charle: When light reflects on an object, a camera observes its brightness, shape, and color. Hyperspectral cameras are very similar but measure the color information in much more detail. Multiple separate color filters on the hyperspectral image sensor decompose the light per wavelength, resulting in the measurement of the object’s spectral fingerprint. This information provides insights into which materials are being observed and also in what they are composed of. In the case of forests and vegetation, relevant observables are chlorophyll (600-700 nm) and water absorption (900-1000 nm and 1400-1600 nm). Not only can the spectral camera see where the vegetation is, but it can also help identify different types of vegetation and soil coverage, as well as providing input to algorithms to measure moist content, which we can imagine being an important parameter to assess fire hazard.
DN: How would this technology be implemented, from a high ranger station tower, a drone, balloon, etc.?
Wouter Charle: A unique advantage of imec’s on-chip spectral imaging technology is that it can provide a real-time video stream of multi-spectral image data. Combined with its small form factor and that these snapshot cameras behave like any other regular machine vision camera, there’s a torrent of different implementation possibilities that enable different use cases. The cameras can, for instance, be mounted on jeeps and helicopters to pro-actively monitor during patrols, where an onboard machine learning system can assign a fire hazard index and upload these data to the cloud. A similar data collection can be realized using piloted or autonomous drones, even more now imec has developed an embedded hyperspectral imaging platform for drone applications. Other implementations could be on towers at specific sensitive locations in forests.
DN: Does hyperspectral imaging have an advantage over current methods, such as the use of traditional cameras, rangers taking soil measurements, and the like?
Wouter Charle: We are not fully aware of the current methods, but typically persons need to go in the field to collect data and analyze in labs or do a visual inspection to detect hazards. These methods are time-consuming, provide limited and sparse sampling, and are prone to errors.
DN: When might hyperspectral technology be available to combat forest fires?
Wouter Charle: The imaging technology exists and is off-the-shelf available for application developers today. All it takes is to start developing the algorithms and the platform to realize this application. Imec has an extended support team of application engineers that help the technology adapters to realize their application, from training in technology adoption over system engineering to data processing and algorithms engineering.
DN: Are there any other interesting applications besides agriculture for this tech?
Wouter Charle: The applications of hyperspectral imaging are virtually unlimited. imec provides both real-time video-rate cameras as high-resolution scanning cameras. These are currently being used by companies and research groups for applications engineering in medical and health, smart farming and agriculture, food, sorting and industrial machine vision, automotive, forest management and reforestation monitoring, environment monitoring, surveillance, etc.
Imec Hyperspectral imaging technology for agricultural crop markets.
John Blyler is a Design News senior editor, covering the electronics and advanced manufacturing spaces. With a BS in Engineering Physics and an MS in Electrical Engineering, he has years of hardware-software-network systems experience as an editor and engineer within the advanced manufacturing, IoT and semiconductor industries. John has co-authored books related to system engineering and electronics for IEEE, Wiley, and Elsevier.