An Exploration of Encoder-Decoder Approaches to Multi-Label Classification for Legal and Biomedical Text
Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review
Documents
- Fulltext
Final published version, 323 KB, PDF document
Standard methods for multi-label text classification largely rely on encoder-only pre-trained language models, whereas encoder-decoder models have proven more effective in other classification tasks. In this study, we compare four methods for multi-label classification, two based on an encoder only, and two based on an encoder-decoder. We carry out experiments on four datasets-two in the legal domain and two in the biomedical domain, each with two levels of label granularity- and always depart from the same pre-trained model, T5. Our results show that encoder-decoder methods outperform encoder-only methods, with a growing advantage on more complex datasets and labeling schemes of finer granularity. Using encoder-decoder models in a non-autoregressive fashion, in particular, yields the best performance overall, so we further study this approach through ablations to better understand its strengths.
Original language | English |
---|---|
Title of host publication | Findings of the Association for Computational Linguistics, ACL 2023 |
Number of pages | 1 |
Publisher | Association for Computational Linguistics (ACL) |
Publication date | 2023 |
Pages | 5828-5843 |
ISBN (Electronic) | 9781959429623 |
DOIs | |
Publication status | Published - 2023 |
Event | 61st Annual Meeting of the Association for Computational Linguistics, ACL 2023 - Toronto, Canada Duration: 9 Jul 2023 → 14 Jul 2023 |
Conference
Conference | 61st Annual Meeting of the Association for Computational Linguistics, ACL 2023 |
---|---|
Land | Canada |
By | Toronto |
Periode | 09/07/2023 → 14/07/2023 |
Sponsor | Bloomberg Engineering, et al., Google Research, Liveperson, Meta, Microsoft |
Series | Proceedings of the Annual Meeting of the Association for Computational Linguistics |
---|---|
ISSN | 0736-587X |
Bibliographical note
Publisher Copyright:
© 2023 Association for Computational Linguistics.
ID: 374650746