Abstract
Detection of fruits and other plant parts using computer vision is an important step in automating tasks like harvesting,
phenotyping, and yield prediction. This kind of image analysis is being used increasingly in agriculture since it is quick,
non-destructive, and can avoid time-consuming and labor-intensive manual measurements. Segmenting tomatoes from
images taken in a production greenhouse is difficult, a.o., because of occlusions, overlap of the fruit color with other
plant parts, and illumination variations. Recent approaches to detecting fruits mostly focus on supervised and deep
learning methods, in which a large number of images need to be annotated to train the algorithm to be able to decide
what is a fruit. While these methods have the advantage of not having to hand-craft discriminative features, annotating
hundreds of images to obtain a workable training set is very labor intensive. In this work, we developed a method for
the detection of tomatoes, which does not require a training dataset containing labelled fruits. We used an Intel
RealSense TM D435 camera which provides pixel registered depth and color images, mounted on a trolley that
autonomously navigates through the greenhouse and images all plants at regular intervals. First, the depth is used to
extract the front row plants from the RGB image. The intensity of fruit pixels is then enhanced by grayscale transforms.
The fruit pixels are then segmented by thresholding. Next, the watershed algorithm is applied to separate contiguous
regions. Finally, regions above a certain size and whose perimeters approximately fit a circle are selected as individual
fruit instances. Experimental results over 123 images acquired in a greenhouse show that this approach could detect
tomatoes with a recall of 0.8 and a precision of 0.6. In addition to being applicable in a practical context in itself, this
approach has potential for use as a tool to generate data to train supervised (deep) machine learning methods.
phenotyping, and yield prediction. This kind of image analysis is being used increasingly in agriculture since it is quick,
non-destructive, and can avoid time-consuming and labor-intensive manual measurements. Segmenting tomatoes from
images taken in a production greenhouse is difficult, a.o., because of occlusions, overlap of the fruit color with other
plant parts, and illumination variations. Recent approaches to detecting fruits mostly focus on supervised and deep
learning methods, in which a large number of images need to be annotated to train the algorithm to be able to decide
what is a fruit. While these methods have the advantage of not having to hand-craft discriminative features, annotating
hundreds of images to obtain a workable training set is very labor intensive. In this work, we developed a method for
the detection of tomatoes, which does not require a training dataset containing labelled fruits. We used an Intel
RealSense TM D435 camera which provides pixel registered depth and color images, mounted on a trolley that
autonomously navigates through the greenhouse and images all plants at regular intervals. First, the depth is used to
extract the front row plants from the RGB image. The intensity of fruit pixels is then enhanced by grayscale transforms.
The fruit pixels are then segmented by thresholding. Next, the watershed algorithm is applied to separate contiguous
regions. Finally, regions above a certain size and whose perimeters approximately fit a circle are selected as individual
fruit instances. Experimental results over 123 images acquired in a greenhouse show that this approach could detect
tomatoes with a recall of 0.8 and a precision of 0.6. In addition to being applicable in a practical context in itself, this
approach has potential for use as a tool to generate data to train supervised (deep) machine learning methods.
| Original language | English |
|---|---|
| Title of host publication | Proceedings of the European Conference on Agricultural Engineering AgEng 2021 |
| Editors | J. C. Barbosa, L.L. Silva, P. Lourenço, A. Sousa, V.F. Cruz, F. Baptista |
| Publisher | EurAgEng, Cranfield |
| Pages | 29-34 |
| Publication status | Published - 4 Jul 2021 |
| Event | European Conference on Agricultural Engineering AgEng 2021 - Evora, Portugal Duration: 4 Jul 2021 → 8 Jul 2021 |
Conference
| Conference | European Conference on Agricultural Engineering AgEng 2021 |
|---|---|
| Country/Territory | Portugal |
| City | Evora |
| Period | 4/07/21 → 8/07/21 |