Affective Relevance: Inferring Emotional Responses via fNIRS Neuroimaging
Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review
Standard
Affective Relevance : Inferring Emotional Responses via fNIRS Neuroimaging. / Ruotsalo, Tuukka; Spapé, Michiel M.; Mäkelä, Kalle; Leiva, Luis A.
SIGIR 2023 - Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, Inc., 2023. p. 1796-1800.Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - GEN
T1 - Affective Relevance
T2 - 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2023
AU - Ruotsalo, Tuukka
AU - Spapé, Michiel M.
AU - Mäkelä, Kalle
AU - Leiva, Luis A.
N1 - Publisher Copyright: © 2023 Copyright held by the owner/author(s).
PY - 2023
Y1 - 2023
N2 - Information retrieval (IR) relies on a general notion of relevance, which is used as the principal foundation for ranking and evaluation methods. However, IR does not account for more a nuanced affective experience. Here, we consider the emotional response decoded directly from the human brain as an alternative dimension of relevance. We report an experiment covering seven different scenarios in which we measure and predict how users emotionally respond to visual image contents by using functional near-infrared spectroscopy (fNIRS) neuroimaging on two commonly used affective dimensions: valence (negativity and positivity) and arousal (bored-ness and excitedness). Our results show that affective states can be successfully decoded using fNIRS, and utilized to complement the present notion of relevance in IR studies. For example, we achieved 0.39 Balanced accuracy and 0.61 AUC in 4-class classification of affective states (vs. 0.25 Balanced accuracy and 0.5 AUC of a random classifier). Likewise, we achieved 0.684 Precision@20 when retrieving high-arousal images. Our work opens new avenues for incorporating emotional states in IR evaluation, affective feedback, and information filtering.
AB - Information retrieval (IR) relies on a general notion of relevance, which is used as the principal foundation for ranking and evaluation methods. However, IR does not account for more a nuanced affective experience. Here, we consider the emotional response decoded directly from the human brain as an alternative dimension of relevance. We report an experiment covering seven different scenarios in which we measure and predict how users emotionally respond to visual image contents by using functional near-infrared spectroscopy (fNIRS) neuroimaging on two commonly used affective dimensions: valence (negativity and positivity) and arousal (bored-ness and excitedness). Our results show that affective states can be successfully decoded using fNIRS, and utilized to complement the present notion of relevance in IR studies. For example, we achieved 0.39 Balanced accuracy and 0.61 AUC in 4-class classification of affective states (vs. 0.25 Balanced accuracy and 0.5 AUC of a random classifier). Likewise, we achieved 0.684 Precision@20 when retrieving high-arousal images. Our work opens new avenues for incorporating emotional states in IR evaluation, affective feedback, and information filtering.
KW - Affective computing
KW - Affective feedback
KW - Emotion detection
U2 - 10.1145/3539618.3591946
DO - 10.1145/3539618.3591946
M3 - Article in proceedings
AN - SCOPUS:85168651392
SP - 1796
EP - 1800
BT - SIGIR 2023 - Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval
PB - Association for Computing Machinery, Inc.
Y2 - 23 July 2023 through 27 July 2023
ER -
ID: 383791513