Multimodal Classification of Remote Sensing Images: A Review and Future Directions

Luis Gomez-Chova, Devis Tuia, Gabriele Moser, Gustau Camps-Valls

Research output: Contribution to journalArticleAcademicpeer-review

127 Citations (Scopus)


Earth observation through remote sensing images allows the accurate characterization and identification of materials on the surface from space and airborne platforms. Multiple and heterogeneous image sources can be available for the same geographical region: multispectral, hyperspectral, radar, multitemporal, and multiangular images can today be acquired over a given scene. These sources can be combined/fused to improve classification of the materials on the surface. Even if this type of systems is generally accurate, the field is about to face new challenges: the upcoming constellations of satellite sensors will acquire large amounts of images of different spatial, spectral, angular, and temporal resolutions. In this scenario, multimodal image fusion stands out as the appropriate framework to address these problems. In this paper, we provide a taxonomical view of the field and review the current methodologies for multimodal classification of remote sensing images. We also highlight the most recent advances, which exploit synergies with machine learning and signal processing: sparse methods, kernel-based fusion, Markov modeling, and manifold alignment. Then, we illustrate the different approaches in seven challenging remote sensing applications: 1) multiresolution fusion for multispectral image classification; 2) image downscaling as a form of multitemporal image fusion and multidimensional interpolation among sensors of different spatial, spectral, and temporal resolutions; 3) multiangular image classification; 4) multisensor image fusion exploiting physically-based feature extractions; 5) multitemporal image classification of land covers in incomplete, inconsistent, and vague image sources; 6) spatiospectral multisensor fusion of optical and radar images for change detection; and 7) cross-sensor adaptation of classifiers. The adoption of these techniques in operational settings will help to monitor our planet from space in the very near future.

Original languageEnglish
Article number7182258
Pages (from-to)1560-1584
Number of pages25
JournalProceedings of the IEEE
Issue number9
Publication statusPublished - Sep 2015
Externally publishedYes


  • Classification
  • fusion
  • multiangular
  • multimodal image analysis
  • multisource
  • multitemporal
  • remote sensing

Fingerprint Dive into the research topics of 'Multimodal Classification of Remote Sensing Images: A Review and Future Directions'. Together they form a unique fingerprint.

  • Cite this