ملخص
Machine learning models have been found to learn shortcuts - unintended decision rules that are unable to generalize - undermining models' reliability. Previous works address this problem under the tenuous assumption that only a single shortcut exists in the training data. Real-world images are rife with multiple visual cues from background to texture. Key to advancing the reliability of vision systems is understanding whether existing methods can overcome multiple shortcuts or struggle in a Whac-A-Mole game, i.e., where mitigating one shortcut amplifies reliance on others. To address this shortcoming, we propose two benchmarks: 1) UrbanCars, a dataset with precisely controlled spurious cues, and 2) ImageNet-W, an evaluation set based on ImageNet for watermark, a shortcut we discovered affects nearly every modern vision model. Along with texture and background, ImageNet-W allows us to study multiple shortcuts emerging from training on natural images. We find computer vision models, including large foundation models - regardless of training set, architecture, and supervision - struggle when multiple shortcuts are present. Even methods explicitly designed to combat shortcuts struggle in a Whac-A-Mole dilemma. To tackle this challenge, we propose Last Layer Ensemble, a simple-yet-effective method to mitigate multiple shortcuts without Whac-A-Mole behavior. Our results surface multi-shortcut mitigation as an overlooked challenge critical to advancing the reliability of vision systems. The datasets and code are released: https://github.com/facebookresearch/Whac-A-Mole.
اللغة الأصلية | الإنجليزيّة |
---|---|
عنوان منشور المضيف | Proceedings - 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023 |
ناشر | IEEE Computer Society |
الصفحات | 20071-20082 |
عدد الصفحات | 12 |
رقم المعيار الدولي للكتب (الإلكتروني) | 9798350301298 |
المعرِّفات الرقمية للأشياء | |
حالة النشر | نُشِر - 2023 |
الحدث | 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023 - Vancouver, كندا المدة: ١٨ يونيو ٢٠٢٣ → ٢٢ يونيو ٢٠٢٣ |
سلسلة المنشورات
الاسم | Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition |
---|---|
مستوى الصوت | 2023-June |
رقم المعيار الدولي للدوريات (المطبوع) | 1063-6919 |
!!Conference
!!Conference | 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023 |
---|---|
الدولة/الإقليم | كندا |
المدينة | Vancouver |
المدة | ١٨/٠٦/٢٣ → ٢٢/٠٦/٢٣ |
ملاحظة ببليوغرافية
Publisher Copyright:© 2023 IEEE.