Fine-grained categorization and dataset bootstrapping using deep metric learning with humans in the loop
Research output: Contribution to journal › Conference article › Research › peer-review
Standard
Fine-grained categorization and dataset bootstrapping using deep metric learning with humans in the loop. / Cui, Yin; Zhou, Feng; Lin, Yuanqing; Belongie, Serge.
In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 09.12.2016, p. 1153-1162.Research output: Contribution to journal › Conference article › Research › peer-review
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - GEN
T1 - Fine-grained categorization and dataset bootstrapping using deep metric learning with humans in the loop
AU - Cui, Yin
AU - Zhou, Feng
AU - Lin, Yuanqing
AU - Belongie, Serge
N1 - Publisher Copyright: © 2016 IEEE.
PY - 2016/12/9
Y1 - 2016/12/9
N2 - Existing fine-grained visual categorization methods often suffer from three challenges: lack of training data, large number of fine-grained categories, and high intraclass vs. low inter-class variance. In this work we propose a generic iterative framework for fine-grained categorization and dataset bootstrapping that handles these three challenges. Using deep metric learning with humans in the loop, we learn a low dimensional feature embedding with anchor points on manifolds for each category. These anchor points capture intra-class variances and remain discriminative between classes. In each round, images with high confidence scores from our model are sent to humans for labeling. By comparing with exemplar images, labelers mark each candidate image as either a 'true positive' or a 'false positive.' True positives are added into our current dataset and false positives are regarded as 'hard negatives' for our metric learning model. Then the model is retrained with an expanded dataset and hard negatives for the next round. To demonstrate the effectiveness of the proposed framework, we bootstrap a fine-grained flower dataset with 620 categories from Instagram images. The proposed deep metric learning scheme is evaluated on both our dataset and the CUB-200-2001 Birds dataset. Experimental evaluations show significant performance gain using dataset bootstrapping and demonstrate state-of-the-art results achieved by the proposed deep metric learning methods.
AB - Existing fine-grained visual categorization methods often suffer from three challenges: lack of training data, large number of fine-grained categories, and high intraclass vs. low inter-class variance. In this work we propose a generic iterative framework for fine-grained categorization and dataset bootstrapping that handles these three challenges. Using deep metric learning with humans in the loop, we learn a low dimensional feature embedding with anchor points on manifolds for each category. These anchor points capture intra-class variances and remain discriminative between classes. In each round, images with high confidence scores from our model are sent to humans for labeling. By comparing with exemplar images, labelers mark each candidate image as either a 'true positive' or a 'false positive.' True positives are added into our current dataset and false positives are regarded as 'hard negatives' for our metric learning model. Then the model is retrained with an expanded dataset and hard negatives for the next round. To demonstrate the effectiveness of the proposed framework, we bootstrap a fine-grained flower dataset with 620 categories from Instagram images. The proposed deep metric learning scheme is evaluated on both our dataset and the CUB-200-2001 Birds dataset. Experimental evaluations show significant performance gain using dataset bootstrapping and demonstrate state-of-the-art results achieved by the proposed deep metric learning methods.
UR - http://www.scopus.com/inward/record.url?scp=84986332657&partnerID=8YFLogxK
U2 - 10.1109/CVPR.2016.130
DO - 10.1109/CVPR.2016.130
M3 - Conference article
AN - SCOPUS:84986332657
SP - 1153
EP - 1162
JO - I E E E Conference on Computer Vision and Pattern Recognition. Proceedings
JF - I E E E Conference on Computer Vision and Pattern Recognition. Proceedings
SN - 1063-6919
T2 - 29th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016
Y2 - 26 June 2016 through 1 July 2016
ER -
ID: 301828374