Learning a Mahalanobis metric from equivalence constraints

Aharon Bar-Hillel, Tomer Hertz, Noam Shental, Daphna Weinshall

Research output: Contribution to journalArticlepeer-review


Many learning algorithms use a metric defined over the input space as a principal tool, and their performance critically depends on the quality of this metric. We address the problem of learning metrics using side-information in the form of equivalence constraints. Unlike labels, we demonstrate that this type of side-information can sometimes be automatically obtained without the need of human intervention. We show how such side-information can be used to modify the representation of the data, leading to improved clustering and classification. Specifically, we present the Relevant Component Analysis (RCA) algorithm, which is a simple and efficient algorithm for learning a Mahalanobis metric. We show that RCA is the solution of an interesting optimization problem, founded on an information theoretic basis. If dimensionality reduction is allowed within RCA, we show that it is optimally accomplished by a version of Fisher's linear discriminant that uses constraints. Moreover, under certain Gaussian assumptions, RCA can be viewed as a Maximum Likelihood estimation of the within class covariance matrix. We conclude with extensive empirical evaluations of RCA, showing its advantage over alternative methods.

Original languageEnglish
Pages (from-to)937-965
JournalJournal of Machine Learning Research
StatePublished - Jun 2005
Externally publishedYes


  • Clustering
  • Dimensionality reduction
  • Equivalence constraints
  • Metric learning
  • Side information


Dive into the research topics of 'Learning a Mahalanobis metric from equivalence constraints'. Together they form a unique fingerprint.

Cite this