TY - JOUR

T1 - Transfer learning in computer vision tasks

T2 - Remember where you come from

AU - Li, Xuhong

AU - Grandvalet, Yves

AU - Davoine, Franck

AU - Cheng, Jingchun

AU - Cui, Yin

AU - Zhang, Hang

AU - Belongie, Serge

AU - Tsai, Yi Hsuan

AU - Yang, Ming Hsuan

N1 - Funding Information: This work was carried out with the supports of the China Scholarship Council and of a PEPS grant through the DESSTOPT project jointly managed by the National Institute of Mathematical Sciences and their Interactions (INSMI) and the Institute of Information Science and their Interactions (INS2I) of the CNRS, France. It was carried out in the framework of SIVALab, a joint laboratory between Renault and Heudiasyc (UTC/CNRS). We acknowledge the support of NVIDIA Corporation with the donation of GPUs used for this research. Funding Information: This work was carried out with the supports of the China Scholarship Council and of a PEPS grant through the DESSTOPT project jointly managed by the National Institute of Mathematical Sciences and their Interactions (INSMI) and the Institute of Information Science and their Interactions (INS2I) of the CNRS, France. It was carried out in the framework of SIVALab, a joint laboratory between Renault and Heudiasyc (UTC/CNRS). We acknowledge the support of NVIDIA Corporation with the donation of GPUs used for this research. Publisher Copyright: © 2019 Elsevier B.V.

PY - 2020

Y1 - 2020

N2 - Fine-tuning pre-trained deep networks is a practical way of benefiting from the representation learned on a large database while having relatively few examples to train a model. This adjustment is nowadays routinely performed so as to benefit of the latest improvements of convolutional neural networks trained on large databases. Fine-tuning requires some form of regularization, which is typically implemented by weight decay that drives the network parameters towards zero. This choice conflicts with the motivation for fine-tuning, as starting from a pre-trained solution aims at taking advantage of the previously acquired knowledge. Hence, regularizers promoting an explicit inductive bias towards the pre-trained model have been recently proposed. This paper demonstrates the versatility of this type of regularizer across transfer learning scenarios. We replicated experiments on three state-of-the-art approaches in image classification, image segmentation, and video analysis to compare the relative merits of regularizers. These tests show systematic improvements compared to weight decay. Our experimental protocol put forward the versatility of a regularizer that is easy to implement and to operate that we eventually recommend as the new baseline for future approaches to transfer learning relying on fine-tuning.

AB - Fine-tuning pre-trained deep networks is a practical way of benefiting from the representation learned on a large database while having relatively few examples to train a model. This adjustment is nowadays routinely performed so as to benefit of the latest improvements of convolutional neural networks trained on large databases. Fine-tuning requires some form of regularization, which is typically implemented by weight decay that drives the network parameters towards zero. This choice conflicts with the motivation for fine-tuning, as starting from a pre-trained solution aims at taking advantage of the previously acquired knowledge. Hence, regularizers promoting an explicit inductive bias towards the pre-trained model have been recently proposed. This paper demonstrates the versatility of this type of regularizer across transfer learning scenarios. We replicated experiments on three state-of-the-art approaches in image classification, image segmentation, and video analysis to compare the relative merits of regularizers. These tests show systematic improvements compared to weight decay. Our experimental protocol put forward the versatility of a regularizer that is easy to implement and to operate that we eventually recommend as the new baseline for future approaches to transfer learning relying on fine-tuning.

KW - Computer vision

KW - Parameter regularization

KW - Transfer learning

U2 - 10.1016/j.imavis.2019.103853

DO - 10.1016/j.imavis.2019.103853

M3 - Journal article

AN - SCOPUS:85076296226

VL - 93

JO - Image and Vision Computing

JF - Image and Vision Computing

SN - 0262-8856

M1 - 103853

ER -