Exploring Visual Engagement Signals for Representation Learning

Research output: Contribution to journal › Conference article › Research › peer-review

Standard

Exploring Visual Engagement Signals for Representation Learning. / Belongie, Serge; Wu, Zuxuan; Cardie, Claire; Jia, Menglin; Reiter, Austin; Lim, Ser-Nam.

In: IEEE Xplore Digital Library, Vol. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 28.02.2022, p. 4186-4197.

Research output: Contribution to journal › Conference article › Research › peer-review

Harvard

Belongie, S, Wu, Z, Cardie, C, Jia, M, Reiter, A & Lim, S-N 2022, 'Exploring Visual Engagement Signals for Representation Learning', IEEE Xplore Digital Library, vol. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 4186-4197. https://doi.org/10.1109/ICCV48922.2021.00417

APA

Belongie, S., Wu, Z., Cardie, C., Jia, M., Reiter, A., & Lim, S-N. (2022). Exploring Visual Engagement Signals for Representation Learning. IEEE Xplore Digital Library, 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 4186-4197. https://doi.org/10.1109/ICCV48922.2021.00417

Vancouver

Belongie S, Wu Z, Cardie C, Jia M, Reiter A, Lim S-N. Exploring Visual Engagement Signals for Representation Learning. IEEE Xplore Digital Library. 2022 Feb 28;2021 IEEE/CVF International Conference on Computer Vision (ICCV):4186-4197. https://doi.org/10.1109/ICCV48922.2021.00417

Author

Belongie, Serge ; Wu, Zuxuan ; Cardie, Claire ; Jia, Menglin ; Reiter, Austin ; Lim, Ser-Nam. / Exploring Visual Engagement Signals for Representation Learning. In: IEEE Xplore Digital Library. 2022 ; Vol. 2021 IEEE/CVF International Conference on Computer Vision (ICCV). pp. 4186-4197.

Bibtex

@inproceedings{bc064f0408924dd680137f4ea40a1df9,
title = "Exploring Visual Engagement Signals for Representation Learning",
abstract = "Visual engagement in social media platforms comprises interactions with photo posts including comments, shares, and likes. In this paper, we leverage such visual engagement clues as supervisory signals for representation learning. However, learning from engagement signals is non-trivial as it is not clear how to bridge the gap between low-level visual information and high-level social interaction. We present VisE,, a weakly supervised learning approach, which maps social images to pseudo labels derived by clustered engagement signals. We then study how models trained in this way benefit subjective downstream computer vision tasks such as emotion recognition or political bias detection. Through extensive studies, we empirically demonstrate the effectiveness of VisE across a diverse set of classification tasks beyond the scope of conventional recognition.",
author = "Serge Belongie and Zuxuan Wu and Claire Cardie and Menglin Jia and Austin Reiter and Ser-Nam Lim",
year = "2022",
month = feb,
day = "28",
doi = "10.1109/ICCV48922.2021.00417",
language = "English",
volume = "2021 IEEE/CVF International Conference on Computer Vision (ICCV)",
pages = "4186--4197",
journal = "IEEE Xplore Digital Library",

}

RIS

TY - GEN

T1 - Exploring Visual Engagement Signals for Representation Learning

AU - Belongie, Serge

AU - Wu, Zuxuan

AU - Cardie, Claire

AU - Jia, Menglin

AU - Reiter, Austin

AU - Lim, Ser-Nam

PY - 2022/2/28

Y1 - 2022/2/28

N2 - Visual engagement in social media platforms comprises interactions with photo posts including comments, shares, and likes. In this paper, we leverage such visual engagement clues as supervisory signals for representation learning. However, learning from engagement signals is non-trivial as it is not clear how to bridge the gap between low-level visual information and high-level social interaction. We present VisE,, a weakly supervised learning approach, which maps social images to pseudo labels derived by clustered engagement signals. We then study how models trained in this way benefit subjective downstream computer vision tasks such as emotion recognition or political bias detection. Through extensive studies, we empirically demonstrate the effectiveness of VisE across a diverse set of classification tasks beyond the scope of conventional recognition.

AB - Visual engagement in social media platforms comprises interactions with photo posts including comments, shares, and likes. In this paper, we leverage such visual engagement clues as supervisory signals for representation learning. However, learning from engagement signals is non-trivial as it is not clear how to bridge the gap between low-level visual information and high-level social interaction. We present VisE,, a weakly supervised learning approach, which maps social images to pseudo labels derived by clustered engagement signals. We then study how models trained in this way benefit subjective downstream computer vision tasks such as emotion recognition or political bias detection. Through extensive studies, we empirically demonstrate the effectiveness of VisE across a diverse set of classification tasks beyond the scope of conventional recognition.

UR - https://openaccess.thecvf.com/content/ICCV2021/html/Jia_Exploring_Visual_Engagement_Signals_for_Representation_Learning_ICCV_2021_paper.html

U2 - 10.1109/ICCV48922.2021.00417

DO - 10.1109/ICCV48922.2021.00417

M3 - Conference article

VL - 2021 IEEE/CVF International Conference on Computer Vision (ICCV)

SP - 4186

EP - 4197

JO - IEEE Xplore Digital Library

JF - IEEE Xplore Digital Library

ER -

ID: 303806039