CMMA: Benchmarking Multi-Affection Detection in Chinese Multi-Modal Conversations
Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review
Documents
- Fulltext
Accepted author manuscript, 1.44 MB, PDF document
Human communication has a multi-modal and multi-affection nature. The inter-relatedness of different emotions and sentiments poses a challenge to jointly detect multiple human affections with multi-modal clues. Recent advances in this field employed multi-task learning paradigms to render the inter-relatedness across tasks, but the scarcity of publicly available resources sets a limit to the potential of works. To fill this gap, we build the first Chinese Multi-modal Multi-Affection conversation (CMMA) dataset, which contains 3,000 multi-party conversations and 21,795 multi-modal utterances collected from various styles of TV-series. CMMA contains a wide variety of affection labels, including sentiment, emotion, sarcasm and humor, as well as the novel inter-correlations values between certain pairs of tasks. Moreover, it provides the topic and speaker information in conversations, which promotes better modeling of conversational context. On the dataset, we empirically analyze the influence of different data modalities and conversational contexts on different affection analysis tasks, and exhibit the practical benefit of inter-task correlations. The full dataset will be publicly available for research\footnote{https://github.com/annoymity2022/Chinese-Dataset}
Original language | English |
---|---|
Title of host publication | Advances in Neural Information Processing Systems 36 pre-proceedings (NeurIPS 2023) |
Number of pages | 12 |
Publisher | NeurIPS Proceedings |
Publication date | 2023 |
Publication status | Published - 2023 |
Event | 37th Conference on Neural Information Processing Systems - NeurIPS 2023 - New Orleans., United States Duration: 10 Dec 2023 → 16 Dec 2023 |
Conference
Conference | 37th Conference on Neural Information Processing Systems - NeurIPS 2023 |
---|---|
Land | United States |
By | New Orleans. |
Periode | 10/12/2023 → 16/12/2023 |
Series | Advances in Neural Information Processing Systems |
---|---|
Volume | 36 |
ISSN | 1049-5258 |
ID: 383796510