PosePAL: Efficient Animal Pose Labeling Using Point Trackers

Automatically estimating animal poses from videos is important for studying animal behavior. Existing methods do not perform reliably, partly because they rely on high-quality training data, and existing such datasets are small. It is challenging and expensive to create such data, due to the large variations in animal morphology and the difficulty of frame-wise video annotation in general. In this paper, we introduce an animal pose labeling pipeline that leverages test-time optimization of a point tracker, providing temporally-consistent annotations at low cost. Given a video, we fine-tune a lightweight appearance embedding inside a pre-trained general-purpose point tracker on a sparse set of annotated frames. These annotations can be obtained from human labelers or off-the-shelf pose detectors. The fine-tuned model is then applied to the rest of the frames for automatic labeling. Our method achieves state-of-the-art performance at a reasonable annotation cost. We believe our pipeline offers a valuable tool for the automatic quantification of animal behavior. Visit our project webpage at https://zhuoyang-pan.github.io/animal-labeling .