An end-to-end method for predicting the respiratory rate of dairy cows from RGB videos

Respiratory rate (RR) is an important indicator of the health and welfare status of dairy cows. In recent years, progress has been made in monitoring the RR of dairy cows using video data. However, existing methods often involve multiple processing modules, such as region of interest detection and tracking, which can introduce errors that propagate through successive steps. The objective of this study was to develop an end-to-end computer vision (CV) method to predict RR of dairy cows continuously and automatically. The method leverages a state-of-the-art Transformer model, VideoMAE, which divides video frames into patches as input tokens, enabling the automated selection and featurization of relevant regions, such as a cow's abdomen, for predicting RR. The original encoder of VideoMAE was retained, and a classification head was added on top of it. The model was fine-tuned and tested on 17 video segments (16.2 ± 11.00 min; Mean ± SD) collected in a tie-stall barn from 6 dairy cows, capturing them resting with minimal movement from top and side views. Respiratory rates measured using a respiratory belt for individual cows were serving as the ground truth. The evaluation of the developed model was conducted using multiple metrics, including mean absolute error of 2.60 breaths per minute (bpm), root mean squared error of 3.62 bpm, root mean squared prediction error (as a proportion of observed mean) of 14.0%, and Pearson correlation of 0.91. The developed CV-based method offers the potential for an end-to-end solution to monitor RR automatically.