Monitoring mammalian herbivores via convolutional neural networks implemented on thermal UAV imagery

Diego Bárbulo Barrios, João Valente*, Frank van Langevelde

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review


Lightweight Unmanned Aerial Vehicles (UAVs) are emerging as a remote sensing survey tool for animal monitoring in several fields, such as precision livestock farming. Together with state-of-the-art computer vision techniques, UAV. technology has drastically escalated our ability to acquire and analyse visual data in the field, lowering both costs and complications associated with collection and analysis. This paper addresses monitoring mammalian herbivores using the unexploited field of thermal Multi-Object Tracking and Segmentation (MOTS) in UAV imagery. In our research, a state-of-the-art MOTS algorithm (Track R-CNN) was trained and evaluated in the segmentation, detection and tracking of dairy cattle. Data collection was carried out in two farms with a UAV carrying a thermal camera at various angles and heights, and under different light (overcast/sunny) and thermal (16.5 °C range) conditions. Our findings suggest that dataset diversity and balance, especially regarding the range of conditions under which the data was collected, can significantly enhance tracking efficiency in specific scenarios. For training the algorithm, transfer learning was used as a knowledge migration method. The performance of our best model (68.5 sMOTSA, 79.6 MOTSA, 41 IDS, 100 % counting accuracy, and 87.2 MOTSP), which utilizes 3D convolutions and an association head, demonstrates the applicability and optimal performance of Track R-CNN in detecting, tracking, and counting herbivores in UAV thermal imagery under heterogenous conditions. Our findings demonstrate that 3D convolutions outperform Long-short Term Memory (LSTM) convolutions. However, LSTM convolutions also show optimal performance, offering a viable alternative. Furthermore, our results highlight the inability of Optical Flow to track motionless animals (-15 sMOTSA, −4.1 MOTSA and 2076 IDS) and the proficiency of the association head in differentiating static animals from the background. This research contributes to the growing body of knowledge in automated mammalian herbivore monitoring, with potential applications such as precision livestock farming and wildlife conservation.

Original languageEnglish
Article number108713
JournalComputers and Electronics in Agriculture
Publication statusPublished - Mar 2024


  • Animal monitoring
  • Deep learning
  • Instance segmentation
  • Livestock
  • Tracking


Dive into the research topics of 'Monitoring mammalian herbivores via convolutional neural networks implemented on thermal UAV imagery'. Together they form a unique fingerprint.

Cite this