Optimising Realism of Synthetic Agricultural Images using Cycle Generative Adversarial Networks

R. Barth, J.M.M. IJsselmuiden, J. Hemming, E.J. van Henten

Research output: Chapter in Book/Report/Conference proceedingConference paper


A bottleneck of state-of-the-art machine learning methods, e.g. deep learning, for plant part image segmentation in agricultural robotics is the requirement of large manually annotated datasets. As a solution, large synthetic datasets including ground truth can be rendered that realistically reflect the empirical situation. However, a dissimilarity gap can remain between synthetic and empirical data by incomplete manual modelling. This paper contributes to closing this gap by optimising the realism of synthetic agricultural images using unsupervised cycle generative adversarial networks, enabling unpaired image-to-image translation from the synthetic to empirical domain and vice versa. For this purpose, the Capsicum annuum (sweet- or bell pepper) dataset was used, containing 10,500 synthetic and 50 empirical annotated images. Additionally, 225 unlabelled empirical images were used. We hypothesised that the similarity of the synthetic images with the empirical images increases qualitatively and quantitively when translated to the empirical domain and investigated the effect of the translation on the factors color, local texture and morphology. Results showed an increased mean class color distribution correlation with the empirical dataset from 0.62 prior and 0.90 post translation of the synthetic dataset. Qualitatively, synthetic images translate very well in local features such as color,
illumination scattering and texture. However, global features like plant morphology appeared not to be translatable.
Original languageEnglish
Title of host publicationProceedings of the IEEE IROS workshop on Agricultural Robotics
Subtitle of host publicationlearning from Industry 4.0 and moving into the future
EditorsTsampikos Kounalakis, Frits van Evert, David Michael Ball, Gert Kootstra, Lazaros Nalpantidis
Place of PublicationWageningen
PublisherWageningen University & Research
Publication statusPublished - 2017

Fingerprint Dive into the research topics of 'Optimising Realism of Synthetic Agricultural Images using Cycle Generative Adversarial Networks'. Together they form a unique fingerprint.

Cite this