Label-Similarity Curriculum Learning
Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review
Documents
- Fulltext
Submitted manuscript, 1.59 MB, PDF document
Curriculum learning can improve neural network training by guiding the optimization to desirable optima. We propose a novel curriculum learning approach for image classification that adapts the loss function by changing the label representation. The idea is to use a probability distribution over classes as target label, where the class probabilities reflect the similarity to the true class. Gradually, this label representation is shifted towards the standard one-hot-encoding. That is, in the beginning minor mistakes are corrected less than large mistakes, resembling a teaching process in which broad concepts are explained first before subtle differences are taught. The class similarity can be based on prior knowledge. For the special case of the labels being natural words, we propose a generic way to automatically compute the similarities. The natural words are embedded into Euclidean space using a standard word embedding. The probability of each class is then a function of the cosine similarity between the vector representations of the class and the true label. The proposed label-similarity curriculum learning (LCL) approach was empirically evaluated using several popular deep learning architectures for image classification tasks applied to five datasets including ImageNet, CIFAR100, and AWA2. In all scenarios, LCL was able to improve the classification accuracy on the test data compared to standard training. Code to reproduce results is available at https://github.com/speedystream/LCL.
Original language | English |
---|---|
Title of host publication | Computer Vision – ECCV 2020 - 16th European Conference, Proceedings |
Editors | Andrea Vedaldi, Horst Bischof, Thomas Brox, Jan-Michael Frahm |
Number of pages | 17 |
Publisher | Springer VS |
Publication date | 2020 |
Pages | 174-190 |
ISBN (Print) | 9783030585259 |
DOIs | |
Publication status | Published - 2020 |
Event | 16th European Conference on Computer Vision, ECCV 2020 - Glasgow, United Kingdom Duration: 23 Aug 2020 → 28 Aug 2020 |
Conference
Conference | 16th European Conference on Computer Vision, ECCV 2020 |
---|---|
Land | United Kingdom |
By | Glasgow |
Periode | 23/08/2020 → 28/08/2020 |
Series | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
---|---|
Volume | 12374 LNCS |
ISSN | 0302-9743 |
- Classification, Curriculum learning, Deep learning, Multi-modal learning
Research areas
ID: 250554869