Learning to Explain: A Model-Agnostic Framework for Explaining Black Box Models

Oren Barkan, Yuval Asher, Amit Eshel, Yehonatan Elisha, Noam Koenigstein

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

We present Learning to Explain (LTX), a model-agnostic framework designed for providing post-hoc explanations for vision models. The LTX framework introduces an 'explainer' model that generates explanation maps, highlighting the crucial regions that justify the predictions made by the model being explained. To train the explainer, we employ a two-stage process consisting of initial pretraining followed by per-instance finetuning. During both stages of training, we utilize a unique configuration where we compare the explained model's prediction for a masked input with its original prediction for the unmasked input. This approach enables the use of a novel counterfactual objective, which aims to anticipate the model's output using masked versions of the input image. Importantly, the LTX framework is not restricted to a specific model architecture and can provide explanations for both Transformer-based and convolutional models. Through our evaluations, we demonstrate that LTX significantly outperforms the current state-of-the-art in explainability across various metrics. Our code is available at: https://github.comLTX-CodeLTX

Original languageEnglish
Title of host publicationProceedings - 23rd IEEE International Conference on Data Mining, ICDM 2023
EditorsGuihai Chen, Latifur Khan, Xiaofeng Gao, Meikang Qiu, Witold Pedrycz, Xindong Wu
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages944-949
Number of pages6
ISBN (Electronic)9798350307887
DOIs
StatePublished - 2023
Externally publishedYes
Event23rd IEEE International Conference on Data Mining, ICDM 2023 - Shanghai, China
Duration: 1 Dec 20234 Dec 2023

Publication series

NameProceedings - IEEE International Conference on Data Mining, ICDM
ISSN (Print)1550-4786

Conference

Conference23rd IEEE International Conference on Data Mining, ICDM 2023
Country/TerritoryChina
CityShanghai
Period1/12/234/12/23

Bibliographical note

Publisher Copyright:
© 2023 IEEE.

Keywords

  • Explainable AI
  • computer vision
  • transformers

Cite this