Effective unconstrained face recognition by combining multiple descriptors and learned background statistics

Lior Wolf, Tal Hassner, Yaniv Taigman

Research output: Contribution to journalArticlepeer-review


Computer vision systems have demonstrated considerable improvement in recognizing and verifying faces in digital images. Still, recognizing faces appearing in unconstrained, natural conditions remains a challenging task. In this paper, we present a face-image, pair-matching approach primarily developed and tested on the Labeled Faces in the Wild (LFW) benchmark that reflects the challenges of face recognition from unconstrained images. The approach we propose makes the following contributions. 1) We present a family of novel face-image descriptors designed to capture statistics of local patch similarities. 2) We demonstrate how unlabeled background samples may be used to better evaluate image similarities. To this end, we describe a number of novel, effective similarity measures. 3) We show how labeled background samples, when available, may further improve classification performance, by employing a unique pair-matching pipeline. We present state-of-the-art results on the LFW pair-matching benchmarks. In addition, we show our system to be well suited for multilabel face classification (recognition) problem, on both the LFW images and on images from the laboratory controlled multi-PIE database.

Original languageEnglish
Article number5674057
Pages (from-to)1978-1990
Number of pages13
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Issue number10
StatePublished - 2011

Bibliographical note

Funding Information:
The authors are grateful to face.com for providing the face alignment system. Lior Wolf is supported by the Israel Science Foundation (Grant No. 1214/06) and The Ministry of Science and Technology Russia-Israel Scientific Research Cooperation. Parts of this manuscript have been published in [1], [2], [3], [4].


  • Face and gesture recognition
  • face recognition
  • image descriptors.
  • similarity measures


Dive into the research topics of 'Effective unconstrained face recognition by combining multiple descriptors and learned background statistics'. Together they form a unique fingerprint.

Cite this