3D object perception and perceptual learning in the RACE project

Miguel Oliveira, Luís Seabra Lopes*, Gi Hyun Lim, S.H. Kasaei, Ana Maria Tomé, Aneesh Chauhan

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

20 Citations (Scopus)


This paper describes a 3D object perception and perceptual learning system developed for a complex artificial cognitive agent working in a restaurant scenario. This system, developed within the scope of the European project RACE, integrates detection, tracking, learning and recognition of tabletop objects. Interaction capabilities were also developed to enable a human user to take the role of instructor and teach new object categories. Thus, the system learns in an incremental and open-ended way from user-mediated experiences. Based on the analysis of memory requirements for storing both semantic and perceptual data, a dual memory approach, comprising a semantic memory and a perceptual memory, was adopted. The perceptual memory is the central data structure of the described perception and learning system. The goal of this paper is twofold: on one hand, we provide a thorough description of the developed system, starting with motivations, cognitive considerations and architecture design, then providing details on the developed modules, and finally presenting a detailed evaluation of the system; on the other hand, we emphasize the crucial importance of the Point Cloud Library (PCL) for developing such system.1

Original languageEnglish
Pages (from-to)614-626
Number of pages13
JournalRobotics and Autonomous Systems
Publication statusPublished - Jan 2016
Externally publishedYes


  • 3D object perception
  • Dual memory systems
  • Interactive learning
  • Open-ended learning
  • Point-Cloud Library


Dive into the research topics of '3D object perception and perceptual learning in the RACE project'. Together they form a unique fingerprint.

Cite this