GAN meets chemometrics: Segmenting spectral images with pixel2pixel image translation with conditional generative adversarial networks

Puneet Mishra*, Ittai Herrmann

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

Abstract

In analytical chemistry, spectral imaging of complex analytical systems is commonly performed. A major task in spectral imaging analysis is to extract signals related to important analytes present in the imaged scene. Hence, the first task of spectral image analysis is to perform an image segmentation to extract the relevant signals to analyze. However, in the chemometric domain, the traditional image segmentation methods are limited either to threshold-based or pixel-wise classification, therefore, no approache uses the contextual information present in the imaging scene. This study presents for the first time a pixel2pixel (p2p) image translation using conditional generative adversarial networks (cGAN) for the segmentation of spectral images. The p2p cGAN trains two neural models simultaneously where one model (generator) learns to segment and the other model learns to detect if the segmentation performed by the generator model is correct. During the process of generation and detection, the model automatically learns to segment the spectral images accurately. As an application of the p2p cGAN, a case of segmenting visible and near-infrared spectral images of plants was presented. Furthermore, as a comparison, threshold-based and pixel-wise image classification based on partial least-square discriminant analysis were also presented. The results showed that the p2p cGAN based image translation performed the best segmentation task with an intersection over union score of 0.95 ​± ​0.04. The advanced new DL based image processing approaches can complement spectral image processing.

Original languageEnglish
Article number104362
JournalChemometrics and Intelligent Laboratory Systems
Volume215
DOIs
Publication statusPublished - 15 Aug 2021

Keywords

  • Artificial intelligence
  • Hyperspectral imagery
  • Image context
  • Semantic segmentation

Fingerprint

Dive into the research topics of 'GAN meets chemometrics: Segmenting spectral images with pixel2pixel image translation with conditional generative adversarial networks'. Together they form a unique fingerprint.

Cite this