#Industry News
Multispectral and hyperspectral cameras expand the scope of industrial imaging
Quality inspection, color inspection, and process monitoring are just a handful of examples of how non-visible imaging components factor into machine vision systems today.
Imaging techniques beyond the visible wavelength enhance machine vision applications.
While multispectral and hyperspectral cameras are deployed in many applications and industries, one area improved by the emergence of these technologies is industrial inspection. Quality inspection of food and beverage products, pharmaceutical product inspection and sorting, color inspection, and process monitoring are just a handful of examples of how non-visible imaging components factor into machine vision systems today.
Multispectral imaging
Two methods to capture images beyond the visible wavelengths are multispectral and hyperspectral imaging. Hyperspectral imaging involves narrow, usually contiguous spectral bands, involving possibly hundreds or thousands of spectra, while multispectral imaging involves spectral bands of varying bandwidths, not necessarily contiguous, but with up to 10 bands strategically selected.
Both methods essentially refer to the same imaging concept, with multispectral covering fewer discrete spectral bands. Multispectral cameras usually acquire several wavelengths in a single image, and techniques vary from using standard broad bandwidth pixel filter architecture to the use of narrow band tunable filters in pixel space. Deployed in industrial applications including materials inspection, semiconductor and electronics inspection, and food quality inspection, multispectral cameras are available in area scan and line scan formats.
Teledyne DALSA (Waterloo, ON, Canada; www.teledynedalsa.com), for example, offers its Linea ML cameras, an honoree in the 2018 Innovators Award program, using 8k x 4 and 16k x 4 quad-linear CMOS sensors with three native RGB channels plus a separate near-infrared (NIR) channel. The cameras are sensitive in the visible (400 to 700 nm) and near infrared (700 to 1000 nm) wavelengths with wafer level dichroic filters and capture independent RGB+NIR images in a single scan. The Linea ML cameras offer a line rate of 75 kHz x 4 or 5 GB/sec throughput via Camera Link HS fiber interface. The cameras offer a line rate of 70 kHz through Camera Link, and feature wafer-level filters for clear spectral channel separation.
Available with 8, 10, or 12-bit output and multiple region of interest mode, the cameras target print inspection, electronics inspection, materials grading, web inspection, color sorting, and food inspection applications, according to the company.
Based on prism technology providing simultaneous capture along a single optical path with no filter wheels or other moving parts, several multispectral cameras (Figure 1) are offered by JAI (San Jose, CA, USA, www.jai.com). For area scan camera users, JAI offers three multispectral cameras in its Fusion series. These 2-CCD sensor models differ only in resolution and data interfaces: AD-080CL (0.8 MPixel, Camera Link interface, 30 fps), AD-080GE (0.8 MPixel, GigE Vision interface, 30 fps), and AD-130GE (1.3 MPixel, GigE Vision interface, 31 fps.) Each camera is based on the same multispectral aspect, in that a Bayer CCD captures visible color images (400 to 700 nm) in one channel, and a monochrome sensor captures near-infrared data (750 to 900+ nm) in a second channel.
For line scan camera users, JAI offers three multispectral cameras in its Sweep+ series. These prism-based, quad-linear cameras provide separate channels for R, G, B, and NIR data. The SW-2001Q-CL is based on a 4 x 2048 CCD pixel array and features a Camera Link interface with 19 kHz line frequency, while the LQ-401CL—also with Camera Link interface—uses a 4 x 4096 CMOS array with 18 kHz line frequency. Featuring a 10GigE interface, the SW-4000Q-10GE uses a 4 x 4096 CMOS array with 72 kHz line frequency.
Lastly, the company also offers a multispectral line scan option in its Wave series, with its WA-1000D-CL, which features two, prism-mounted InGaAs line sensors (2 x 1024 pixels, Camera Link interface, 39 kHz line frequency). One channel covers the upper NIR spectrum and lower SWIR band (900 to 1400 nm), while the second channel falls in the upper portion of the SWIR band from 1400 to 1700 nm.
“By utilizing different narrow-band light sources in the NIR and SWIR range, along with image fusion techniques, this camera can be used to detect and sort hard-to-differentiate substances, particularly in applications such as food inspection and plastics recycling,” says Rich Dickerson, Manager, Marketing Communications, JAI.
Also developing multispectral cameras by manufacturing filters and attaching micro patterned filters directly to the image sensors through an active alignment system, Salvo Technologies (Previously PIXELTEQ; Seminole, FL, USA; www.opticalfiltershop.com), offers a range of multispectral and polarimetric imagers. SpectroCam series cameras, which are available in ultraviolet, VIS, and SWIR versions, are based on a continuously rotating filter wheel comprised of six to eight interchangeable optical filters. The UV and VIS versions—which cover 200 to 900 nm and 400 to 1000 nm respectively—are based on CCD image sensors, while the SWIR versions use InGaAs sensors.
PixelCam multispectral cameras provide multispectral imaging capabilities from three to nine spectral bands, at up to 30 fps. All three models are based on CCD sensors (4 or 8 MPixel) with custom dichroic filter arrays integrated into the focal plane array at wafer level that extract high-contrast spectral information at specific visible and infrared wavelengths, according to the company. These cameras are sensitive in the 400 to 1000 nm range and are available in either GigE or CoaXPress versions with frame rates up to 15 fps.
Spectral Devices (London, ON, Canada; www.spectraldevices.com) offers two types of multispectral cameras, snapshot and line scan. The company’s line scan cameras are based on the 2 MPixel CMV2000 global shutter CMOS image sensor from ams (Premstaetten, Austria; www.ams.com) and are offered in three standard four-band cameras, as well as custom camera models ranging from 2 to 16 different bands. These cameras target applications such as food quality assurance and inspection, and wafer inspection.
The snapshot cameras are based on the 4 MPixel CMV4000 CMOS image sensor, also developed by ams, and are designed for simultaneous capture of a scene at multiple bands. Offered in six standard models—as well as custom models—the cameras capture anywhere from 2 to 16 bands at speeds of up to 94 fps at full frame rate. These multispectral cameras, according to the company, are suitable for use in applications such as robotics, food processing, and color measurement.
On the sensor side, imec (Leuven, Belgium; www.imec-int.com) developed a multispectral time delay integration (TDI) image sensor called Argus, which is based on CCD-in-CMOS technology. The sensors use a format with 4096 columns and 256 stages per CCD array (or band), with a 5.4 µm pixel size. Additionally, a version with seven bands is available, which enables users to add seven spectral filters.
These prototypes integrate CMOS drivers and readout circuitry and achieve a line rate of up to 300 kHz. When combined with spectral filters, multispectral TDI imaging is possible, and with a customized number of bands and TDI stages. Color or spectral filters can be post-processed on the wafer or cover glass lid.
Hyperspectral imaging
To enable hyperspectral imaging, imec also created off-the-shelf hyperspectral image sensors based on wafers applied directly on top of pixels (Figure 2) on the CMV2000 CMOS image sensor from ams. These image sensors are available in snapshot mosaic, snapshot tiled, line scan wedge, and line scan CCD time delay integration (TDI) formats and offer options with 4, 7, 16, 25, 32, 100+, and 150+ bands. The image sensors are integrated into several machine vision camera models, all of which are suitable for use in various industrial inspection applications.
XIMEA (Münster, Germany; www.ximea.com) offers four models based on imec sensors, including two mosaic-tiled sensors with 16 and 25 bands, and two line scan models with 100 and 150 bands. These cameras feature either USB3 interface with speeds of 170 fps or PCIe with up to 340 fps, and spectral ranges of RGB+NIR, 470 to 630 nm, 600 to 950 nm, 600 to 975 nm, and 470 to 900 nm, depending on the model.
“By applying narrow-band spectral filters at the pixel-level using semiconductor thin-film processing, imec’s technology enables hyperspectral image sensor solutions to reduce form factor, lower weight, and be suitable for embedded vision systems,” says Ivan Klimkovic, Key Account Manager at XIMEA. “XIMEA coupled imec’s hyperspectral sensors with its xiQ camera platform, which complements the important size factor by offering 26.4 x 26.4 x 31 mm in dimension and mere 31 grams in weight.”
Photonfocus (Lachen, Switzerland; www.photonfocus.com) offers three hyperspectral cameras featuring imec sensors as well. Available in snapshot mosaic formats, these cameras offer options with 16 or 25 bands. With a GigE interface, the cameras offer speeds up to 50 fps, and spectral ranges of 470 to 630 nm, 470 to 900 nm, 595 to 860 nm, 600 to 975 nm, and 665 to 975 nm, depending on the model.
Additionally, imec partnered with Adimec (Eindhoven, The Netherlands; www.adimec.com) to develop the imec VNIR hyperspectral system, which is based on an Adimec Quartz machine vision camera with a 2 MPixel CMOS image sensor. Offering a line scan format with 150+ bands and Camera Link interface, this system features a spectral range of 470 to 900 nm or 600 to 1000 nm.
Imec also has its own hyperspectral products, including the SNAPSCAN NIR, SNAPSCAN VNIR, and SNAPSCAN SWIR systems, which feature a USB 3.0 interface and offer snapshot and line scan formats with 100+ and 150+ bands, as well as 470 to 900 nm, 600 to 1000 nm, and 1100 to 1700 nm spectral ranges, depending on the model.
Numerous machine vision companies also develop hyperspectral cameras outside of the imec realm, including Specim (Oulu, Finland; www.specim.fi), which offers the FX series of cameras. These hyperspectral cameras work in line scan mode, are available in GigE, Camera Link, or custom Ethernet, and are designed specifically for industrial machine vision applications.
The FX50 camera (Figure 3) is based on a cooled InSb detector and features a spatial resolution of 640 pixels, a spectral range of 2.7 to 5.3 µm, image acquisition speed of 380 fps, and free wavelength selection from 154 bands within the camera coverage. This camera, according to the company, is suitable for use in black plastic sorting and in detecting contamination on metal surfaces.
“Plastic waste is a huge problem: It is estimated that by 2050, there will be more plastic in the oceans than fish. The majority of non-recyclable plastics consists of mixed plastic types that cannot be reused because traditional plastic sorting technologies are not fulfilling industrial requirements to separate them reliably and efficiently enough,” says Hannu Mäki-Marttunen, Head of Sales & Marketing at Specim. “This is where Specim FX series hyperspectral cameras step in. With the FX17 and the brand-new FX50, we can now identify and sort different plastics, even black plastics, up to 99% accuracy.”
He continues, “This means that Specim end-customers can now transform plastic waste into a valuable resource that can be reused as raw material for the plastics industry.”
The FX17 model is an InGaAs-based camera with a spectral range of 900 to 1700 nm, image acquisition speed of 670 fps, and free wavelength selection from 224 bands within the camera coverage. Target applications include food and feed quality, waste sorting, recycling, and moisture measurement. The FX10 model is a CMOS image sensor-based camera with a spectral range of 400 to 1000 nm, image acquisition speed of 330 fps, and free wavelength selection from 224 bands within the camera coverage. This camera, according to the company, targets machine vision applications such as food quality inspection and detecting color/density in printing applications.
Specim offers a number of other hyperspectral cameras, including the Fenix, PFD-65-V10E, and sCMOS-50-V10E models, as well as the portable Specim IQ camera, which enables mobile material analysis and offers a spectral range of 400 to 1000 nm.
HinaLea Imaging (Kapolei, HI, USA; www.hinaleaimaging.com) also makes hyperspectral cameras, including the Model 4200 wide-field camera, which has 2.3 MPixel sensor spatial resolution, sensitivity in the 400 to 1000 nm range, and access to up to 600 spectral bands. The company also offers the handheld Model 4100H, which reportedly delivers 2.3 MPixel data cubes at up to 550 spectral bands in the visible and near-infrared wavelengths (400 to 1000 nm). The device also has an embedded processor and built-in lighting.
“We’re getting tremendous and growing interest in our cameras for food safety inspection and semiconductor processing,” says Alexandre Fong, Vice President, Engineering. “With the introduction of cost-effective spectral solutions, there is potential to transform automated inspection applications with this depth of new information.”
Targeting advanced machine vision applications as one of its core markets, Headwall Photonics (Bolton, MA, USA; www.headwallphotonics.com) is another company developing spectral cameras. For its Micro-Hyperspec camera, for example, the company lists machine vision as a target application. This camera is available in VNIR, NIR, extended NIR, and SWIR versions, all of which offer Camera Link interface: VNIR A-Series (400 to 1000 nm, silicon CCD sensor, 324 selectable spectral bands, 90 fps); VNIR E-Series (400 to 1000 nm, sCMOS sensor, 369 selectable spectral bands, 250 fps); NIR 640 (900 to 1700 nm, InGaAs detector, 134 selectable spectral bands, 120 fps); NIR 320 (900 to 1700 nm, InGaAs detector, 67 selectable spectral bands, 346 fps); Extended VNIR 640 (600 to 1700 nm, InGaAs detector, 267 selectable spectral bands, 120 fps); SWIR 384 (900 to 2500 nm, MCT detector, 166 selectable spectral bands, 450 fps); and SWIR 640 (900 to 2500 nm, MCT detector, 267 selectable spectral bands, >200 fps).
Designed solely for machine vision applications is the company’s Hyperspec MV camera, which has a wavelength range of 400 to 1000 nm, 270 selectable spectral bands, a Camera Link interface, and an image acquisition speed of 485 fps.
While BaySpec (San Jose, CA, USA; www.bayspec.com), is a company that develops spectral instruments for industries including research and development, biomedical, and optical telecommunications, the company also has hyperspectral cameras that are suitable for industrial inspection. One such camera is the USB 3.0-based OCI-OEM camera, which serves as the optical engine of the company’s OCI-1000 (push-broom, up to 120 fps) and OCI-2000 (snapshot, up to 120 fps) hyperspectral imagers, which cover the 600 to 1000 nm range with up to 100 (OCI-1000) or 25 (OCI-2000) selectable spectral bands.
Another option is the company’s GoldenEye Snapshot hyperspectral camera, which uses proprietary FT-PI technology and covers an extended 400 to 1700 nm range, features 40 to 52 selectable spectral bands, and a frame rate of 1 fps at 648 x 488 spatial pixels.
Similarly, Resonon (Bozeman, MT, USA; www.resonon.com) is a company developing hyperspectral cameras for laboratory, outdoor, and remote sensing applications, while also keeping an eye toward the machine vision market. Suitable for use in industrial imaging applications are the following cameras, according to the company: Pika L (Figure 4; 400 to 1000 nm spectral range, 281 selectable spectral bands, 249 fps, USB 3.0 interface), Pika XC2 (400 to 1000 nm spectral range, 447 selectable spectral bands, 165 fps, USB 3.0 interface), Pika NIR-320 (900 to 1700 nm spectral range, 164 selectable spectral bands, 520 fps, GigE interface), and Pika NIR-640 (900 to 1700 nm, 328 selectable spectral bands, 249 fps, GigE interface.)
Lastly, offering two hyperspectral imaging cameras for industrial imaging applications in its HySpex line is Norsk Elektro Optikk (NEO; Skedsmokorset, Norway; www.hyspex.no). The HySpex SWIR-384 camera is based on an MCT sensor and offers a spectral range of 950 to 2500 nm with 288 selectable spectral bands and a frame rate of 400 fps at full spectral range (scalable by reducing the range), while the HySpex VNIR-1024 is based on a CMOS image sensor and offers a spectral range of 400 to 1000 nm with 108 selectable spectral bands and a frame rate of 700 fps at full spectral resolution.
Both of the hyperspectral cameras, according to the company, are extremely sharp—both spectrally and spatially—with less than 10% spatial and spectral misregistration (smile and keystone).
