TY - JOUR
T1 - Simple Transferability Estimation for Regression Tasks
AU - Nguyen, Cuong N.
AU - Tran, Phong
AU - Ho, Lam Si Tung
AU - Dinh, Vu
AU - Tran, Anh T.
AU - Hassner, Tal
AU - Nguyen, Cuong V.
N1 - Publisher Copyright:
© UAI 2023. All rights reserved.
PY - 2023
Y1 - 2023
N2 - We consider transferability estimation, the problem of estimating how well deep learning models transfer from a source to a target task. We focus on regression tasks, which received little previous attention, and propose two simple and computationally efficient approaches that estimate transferability based on the negative regularized mean squared error of a linear regression model. We prove novel theoretical results connecting our approaches to the actual transferability of the optimal target models obtained from the transfer learning process. Despite their simplicity, our approaches significantly outperform existing state-of-the-art regression transferability estimators in both accuracy and efficiency. On two large-scale keypoint regression benchmarks, our approaches yield 12% to 36% better results on average while being at least 27% faster than previous state-of-the-art methods.
AB - We consider transferability estimation, the problem of estimating how well deep learning models transfer from a source to a target task. We focus on regression tasks, which received little previous attention, and propose two simple and computationally efficient approaches that estimate transferability based on the negative regularized mean squared error of a linear regression model. We prove novel theoretical results connecting our approaches to the actual transferability of the optimal target models obtained from the transfer learning process. Despite their simplicity, our approaches significantly outperform existing state-of-the-art regression transferability estimators in both accuracy and efficiency. On two large-scale keypoint regression benchmarks, our approaches yield 12% to 36% better results on average while being at least 27% faster than previous state-of-the-art methods.
UR - http://www.scopus.com/inward/record.url?scp=85170030262&partnerID=8YFLogxK
M3 - ???researchoutput.researchoutputtypes.contributiontojournal.conferencearticle???
AN - SCOPUS:85170030262
SN - 2640-3498
VL - 216
SP - 1510
EP - 1521
JO - Proceedings of Machine Learning Research
JF - Proceedings of Machine Learning Research
T2 - 39th Conference on Uncertainty in Artificial Intelligence, UAI 2023
Y2 - 31 July 2023 through 4 August 2023
ER -