Abnormal turkey behaviour detection using self-supervised video anomaly detection and multiple object tracking

Recently, researchers have shown an increased interest in the automated visual monitoring of on-farm animal behaviour because of its improved objectivity and efficiency compared to human observations. However, the involved annotation time is a major challenge with video camera data. To reduce the annotation time and automatically detect abnormal behaviours, we develop and train a self-supervised video anomaly detection model based on (optical) flow reconstruction and frame prediction, in order to select frames with abnormal behaviour and identify turkeys by multi-object tracking (MOT). The proposed algorithm first detects turkeys using a you-only-look-once-X detection model and extracts the optical flow in each frame by FlowNet. To track and identify each individual turkey, we use an MOT model based on the ByteTrack algorithm, where we include a third association step based on the turkey head area. Afterwards, the self-supervised anomaly detection model HF²-VAD is employed to detect instances of abnormal behaviour in turkeys. To evaluate the proposed abnormal behaviour detection model, it is tested on 7 videos which result in an area under the curve of 92.1%. Additionally, the proposed MOT model is tested with four 2.5-minute videos and one 5-minute video, obtaining 87.7% MOTA, 80.4% MOTP, 90.8% IDF1 and 72.0% HOTA scores on average for the 2.5-minute videos, and 85.4% MOTA, 82.6% MOTP, 89.1% IDF1 and 72.5% HOTA scores for the 5-minute video, respectively. The results show that the proposed model can successfully detect abnormal behaviour and identify turkeys in new unseen data.