Learning Pose-Aware Models for Pose-Invariant Face Recognition in the Wild

Iacopo Masi, Feng Ju Chang, Jongmoo Choi, Shai Harel, Jungyeon Kim, Kanggeon Kim, Jatuporn Leksut, Stephen Rawls, Yue Wu, Tal Hassner, Wael AbdAlmageed, Gerard Medioni, Louis Philippe Morency, Prem Natarajan, Ram Nevatia

Research output: Contribution to journalArticlepeer-review


We propose a method designed to push the frontiers of unconstrained face recognition in the wild with an emphasis on extreme out-of-plane pose variations. Existing methods either expect a single model to learn pose invariance by training on massive amounts of data or else normalize images by aligning faces to a single frontal pose. Contrary to these, our method is designed to explicitly tackle pose variations. Our proposed Pose-Aware Models (PAM) process a face image using several pose-specific, deep convolutional neural networks (CNN). 3D rendering is used to synthesize multiple face poses from input images to both train these models and to provide additional robustness to pose variations at test time. Our paper presents an extensive analysis of the IARPA Janus Benchmark A (IJB-A), evaluating the effects that landmark detection accuracy, CNN layer selection, and pose model selection all have on the performance of the recognition pipeline. It further provides comparative evaluations on IJB-A and the PIPA dataset. These tests show that our approach outperforms existing methods, even surprisingly matching the accuracy of methods that were specifically fine-tuned to the target dataset. Parts of this work previously appeared in [1] and [2].

Original languageEnglish
Article number8255649
Pages (from-to)379-393
Number of pages15
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Issue number2
StatePublished - 1 Feb 2019

Bibliographical note

Funding Information:
This research is based upon work supported in part by the Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA), via IARPA 2014-14071600010. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of ODNI, IARPA, or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purpose notwithstanding any copyright annotation thereon. Moreover, we gratefully acknowledge USC HPC for hyper-computing and the support of NVIDIA Corporation for their donation of an NVIDIA Titan X. I.M. is the lead author; F.C., J.C., S.H., J.K., K.K., J.L., S.R., and Y.W. in alphabetical order; T.H. is the lead senior author; all remaining senior authors appear next in alphabetical order.

Publisher Copyright:
© 1979-2012 IEEE.


  • CNN
  • Face recognition
  • pose-aware


Dive into the research topics of 'Learning Pose-Aware Models for Pose-Invariant Face Recognition in the Wild'. Together they form a unique fingerprint.

Cite this