UAV-based Semantic Localization and Mapping in Woody Plants

The conventional localization and mapping algorithms are usually based on low-level geometric features (Bowman et al., 2017). These approaches consider only the position and orientation of features and cannot label them with semantic information. In agriculture, the ability for robots to characterize objects semantically is particularly important. Persons can distinguish obstacles from transversal natural agents such as plants, but robots with traditional mapping approaches tend to consider every object as an obstacle (Matsuzaki et al., 2018). Also, semantics' use allows the creation of precise maps with useful information that farmers can employ to monitor and actuate in certain areas of the cultivar. Localization algorithms can also benefit from the use of semantic information. This characterization can improve data association algorithms considering the constraints imposed by the feature’s information (Chebrolu et al., 2019). Besides, different weights, either in filter- and optimization-based algorithms, can be assigned to different categories of semantic features. For example, they can be characterized as static or dynamic, and this can be used to tune the inter-feature confidence in localization algorithms. This research will use lightweight deep learning models to detect fruits and deploy the models in real-time from add source. A particle filter algorithm will use semantic information to localize the UAV. With a precise localization, semantic maps will be generated to provide a characterization of agricultural environments. The proposed approach will be tested using real data in apple orchards and vineyards.