Refining Fidelity Metrics for Explainable Recommendations

Counterfactual evaluation provides a promising framework for assessing explanation fidelity in recommender systems, but perturbation metrics adapted from computer vision suffer three key limitations: (1) they conflate explaining and contradictory features, (2) they average over entire user histories instead of prioritizing concise, high-impact explanations, and (3) they use fixed-percentage perturbations, leading to inconsistencies across users. We introduce refined counterfactual metrics that focus on the most relevant explaining features, exclude contradictory elements, and assess fidelity at a fixed explanation length, ensuring a more consistent and interpretable evaluation. Our code is at: https://github.com/DeltaLabTLV/FidelityMetrics4XRec