Lameness Detection in Dairy Cows from Overhead View: High-Precision Keypoint Localization and Multi-Feature Fusion Classification

Weijun Duan¹Fang Wang¹Honghui Li^1*Na Liu²Xueliang Fu^1*

• ¹College of Computer and Information Engineering, Inner Mongolia Agricultural University, Hohhot, China
• ²College of Animal Science, Inner Mongolia Agricultural University, Hohhot, China

Detecting lameness in dairy cows from an overhead view can effectively avoid occlusion caused by the surrounding facilities or other cows' bodies. Furthermore, the detection equipment is suspended, allowing for parallel detection that does not interfere with the cows' natural behaviors or disrupt the operational order of the farm, thereby better meeting the application requirements of the production environment. However, existing overhead view lameness detection methods still encounter challenges regarding detection accuracy and model generalization capabilities due to the lack of distinct back movement characteristics associated with lameness and the individual variability in movement changes. To address these issues, this paper proposes a study on lameness detection in dairy cows based on RGB-D data from an overhead view. First, we propose a method for detecting key points on the backs of cows from an overhead view, which enhances the accuracy of keypoint localization in complex environments by modeling long-range feature relationships and optimizing the spatial structure of keypoints. Second, we design six lameness features and analyze the correlation in classifying sound, mild lameness, and severe lameness. Finally, we employ the Gini importance index from Random Forest to assess the significance of these features and introduce the PIMP correction method to establish an unbiased feature screening system, enabling accurate detection of lameness behaviors and their severity. The experimental results indicate that the PCK@0.02 and Average Precision (AP) of the keypoint detection network achieved 100.00% and 95.89%, respectively, significantly improving over the baseline model. Additionally, the analysis of lameness features reveals that back curvature, movement asymmetry index, back vertical oscillation, and head vertical oscillation demonstrate excellent discriminative performance in cow lameness classification when employing a multi-feature fusion method. Furthermore, the multi-feature fusion-based model attained a lameness detection accuracy of 91.00%. These research findings provide robust theoretical and technical support for cow health monitoring and intelligent farm management.