Hubs and Hyperspheres: Reducing Hubness and Improving Transductive Few-Shot Learning with Hyperspherical Embeddings

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

Standard

Hubs and Hyperspheres : Reducing Hubness and Improving Transductive Few-Shot Learning with Hyperspherical Embeddings. / Trosten, Daniel J.; Chakraborty, Rwiddhi; Loksc, Sigurd; Wickstrøm, Kristoffer Knutsen; Jenssen, Robert; Kampffmeyer, Michael C.

Proceedings - 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023. IEEE Computer Society Press, 2023. p. 7527-7536.

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

Harvard

Trosten, DJ, Chakraborty, R, Loksc, S, Wickstrøm, KK, Jenssen, R & Kampffmeyer, MC 2023, Hubs and Hyperspheres: Reducing Hubness and Improving Transductive Few-Shot Learning with Hyperspherical Embeddings. in Proceedings - 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023. IEEE Computer Society Press, pp. 7527-7536, 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, Canada, 18/06/2023. https://doi.org/10.1109/CVPR52729.2023.00727

APA

Trosten, D. J., Chakraborty, R., Loksc, S., Wickstrøm, K. K., Jenssen, R., & Kampffmeyer, M. C. (2023). Hubs and Hyperspheres: Reducing Hubness and Improving Transductive Few-Shot Learning with Hyperspherical Embeddings. In Proceedings - 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023 (pp. 7527-7536). IEEE Computer Society Press. https://doi.org/10.1109/CVPR52729.2023.00727

Vancouver

Trosten DJ, Chakraborty R, Loksc S, Wickstrøm KK, Jenssen R, Kampffmeyer MC. Hubs and Hyperspheres: Reducing Hubness and Improving Transductive Few-Shot Learning with Hyperspherical Embeddings. In Proceedings - 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023. IEEE Computer Society Press. 2023. p. 7527-7536 https://doi.org/10.1109/CVPR52729.2023.00727

Author

Trosten, Daniel J. ; Chakraborty, Rwiddhi ; Loksc, Sigurd ; Wickstrøm, Kristoffer Knutsen ; Jenssen, Robert ; Kampffmeyer, Michael C. / Hubs and Hyperspheres : Reducing Hubness and Improving Transductive Few-Shot Learning with Hyperspherical Embeddings. Proceedings - 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023. IEEE Computer Society Press, 2023. pp. 7527-7536

Bibtex

@inproceedings{b06c7eb31c1140a38e51bac3e02bc8e7,
title = "Hubs and Hyperspheres: Reducing Hubness and Improving Transductive Few-Shot Learning with Hyperspherical Embeddings",
abstract = "Distance-based classification is frequently used in transductive few-shot learning (FSL). However, due to the high-dimensionality of image representations, FSL classifiers are prone to suffer from the hubness problem, where a few points (hubs) occur frequently in multiple nearest neighbour lists of other points. Hubness negatively impacts distance-based classification when hubs from one class appear often among the nearest neighbors of points from another class, degrading the classifier's performance. To address the hubness problem in FSL, we first prove that hubness can be eliminated by distributing representations uniformly on the hypersphere. We then propose two new approaches to embed representations on the hypersphere, which we prove optimize a tradeoff between uniformity and local similarity preservation - reducing hubness while retaining class structure. Our experiments show that the proposed methods reduce hubness, and significantly improves transductive FSL accuracy for a wide range of classifiers 11Code available at https://github.com/uitml/noHub..",
keywords = "continual, low-shot, meta, or long-tail learning, Transfer",
author = "Trosten, {Daniel J.} and Rwiddhi Chakraborty and Sigurd Loksc and Wickstr{\o}m, {Kristoffer Knutsen} and Robert Jenssen and Kampffmeyer, {Michael C.}",
note = "Publisher Copyright: {\textcopyright} 2023 IEEE.; 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023 ; Conference date: 18-06-2023 Through 22-06-2023",
year = "2023",
doi = "10.1109/CVPR52729.2023.00727",
language = "English",
pages = "7527--7536",
booktitle = "Proceedings - 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023",
publisher = "IEEE Computer Society Press",
address = "United States",

}

RIS

TY - GEN

T1 - Hubs and Hyperspheres

T2 - 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023

AU - Trosten, Daniel J.

AU - Chakraborty, Rwiddhi

AU - Loksc, Sigurd

AU - Wickstrøm, Kristoffer Knutsen

AU - Jenssen, Robert

AU - Kampffmeyer, Michael C.

N1 - Publisher Copyright: © 2023 IEEE.

PY - 2023

Y1 - 2023

N2 - Distance-based classification is frequently used in transductive few-shot learning (FSL). However, due to the high-dimensionality of image representations, FSL classifiers are prone to suffer from the hubness problem, where a few points (hubs) occur frequently in multiple nearest neighbour lists of other points. Hubness negatively impacts distance-based classification when hubs from one class appear often among the nearest neighbors of points from another class, degrading the classifier's performance. To address the hubness problem in FSL, we first prove that hubness can be eliminated by distributing representations uniformly on the hypersphere. We then propose two new approaches to embed representations on the hypersphere, which we prove optimize a tradeoff between uniformity and local similarity preservation - reducing hubness while retaining class structure. Our experiments show that the proposed methods reduce hubness, and significantly improves transductive FSL accuracy for a wide range of classifiers 11Code available at https://github.com/uitml/noHub..

AB - Distance-based classification is frequently used in transductive few-shot learning (FSL). However, due to the high-dimensionality of image representations, FSL classifiers are prone to suffer from the hubness problem, where a few points (hubs) occur frequently in multiple nearest neighbour lists of other points. Hubness negatively impacts distance-based classification when hubs from one class appear often among the nearest neighbors of points from another class, degrading the classifier's performance. To address the hubness problem in FSL, we first prove that hubness can be eliminated by distributing representations uniformly on the hypersphere. We then propose two new approaches to embed representations on the hypersphere, which we prove optimize a tradeoff between uniformity and local similarity preservation - reducing hubness while retaining class structure. Our experiments show that the proposed methods reduce hubness, and significantly improves transductive FSL accuracy for a wide range of classifiers 11Code available at https://github.com/uitml/noHub..

KW - continual

KW - low-shot

KW - meta

KW - or long-tail learning

KW - Transfer

UR - http://www.scopus.com/inward/record.url?scp=85171965171&partnerID=8YFLogxK

U2 - 10.1109/CVPR52729.2023.00727

DO - 10.1109/CVPR52729.2023.00727

M3 - Article in proceedings

AN - SCOPUS:85171965171

SP - 7527

EP - 7536

BT - Proceedings - 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023

PB - IEEE Computer Society Press

Y2 - 18 June 2023 through 22 June 2023

ER -

ID: 371288564