Interspecies Knowledge Transfer for Facial Keypoint Detection

Main idea. (a) Directly finetuning a human keypoint detector to horses can be suboptimal, since horses and humans have very different shapes and appearances. (b) By warping a horse to have a more human-like shape, the pre-trained human keypoint detector can more easily adapt to the horse’s appearance.

Presenting in CVPR 2017

People

• Maheen Rashid
• Xiuye Gu
• Yong Jae Lee

Abstract

We present a method for localizing facial keypoints on animals by transferring knowledge gained from human faces. Instead of directly finetuning a network trained to detect keypoints in human faces to animal faces (which is sub-optimal since human and animal faces can look quite different), we propose to first adapt the animal images to the pre-trained human detection network by correcting for the differences in animal and human face shape. We first find the nearest human neighbors for each animal image using an unsupervised shape matching method. We use these matches to train a thin plate spline warping network to warp each animal face to look more human-like. The warping network is then jointly finetuned with a pre-trained human facial keypoint detection network using an animal dataset. We demonstrate state-of-the-art results on both horse and sheep facial keypoint detection, and significant improvement over simple finetuning, especially when training data is scarce. Additionally, we present a new dataset with 3717 images with horse face and facial keypoint annotations.

Paper

• Maheen Rashid, Xiuye Gu, Yong Jae Lee
• Interspecies Knowledge Transfer for Facial Keypoint Detection
• CVPR 2017
• [Show BibTex]

@inproceedings{rashid2017interspecies,
  title={Interspecies Knowledge Transfer for Facial Keypoint Detection},
  author={Rashid, Maheen and Gu, Xiuye and Lee, Yong Jae},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  year={2017}
}
          

Source Code

Data and Code are available here

Press

Acknowledgments

This work was supported in part by a gift from the Swedish University of Agricultural Sciences and GPUs donated by NVIDIA.

Contact

Maheen Rashid