An Exploration of Encoder-Decoder Approaches to Multi-Label Classification for Legal and Biomedical Text

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

Documents

  • Fulltext

    Final published version, 323 KB, PDF document

Standard methods for multi-label text classification largely rely on encoder-only pre-trained language models, whereas encoder-decoder models have proven more effective in other classification tasks. In this study, we compare four methods for multi-label classification, two based on an encoder only, and two based on an encoder-decoder. We carry out experiments on four datasets-two in the legal domain and two in the biomedical domain, each with two levels of label granularity- and always depart from the same pre-trained model, T5. Our results show that encoder-decoder methods outperform encoder-only methods, with a growing advantage on more complex datasets and labeling schemes of finer granularity. Using encoder-decoder models in a non-autoregressive fashion, in particular, yields the best performance overall, so we further study this approach through ablations to better understand its strengths.

Original languageEnglish
Title of host publicationFindings of the Association for Computational Linguistics, ACL 2023
Number of pages1
PublisherAssociation for Computational Linguistics (ACL)
Publication date2023
Pages5828-5843
ISBN (Electronic)9781959429623
DOIs
Publication statusPublished - 2023
Event61st Annual Meeting of the Association for Computational Linguistics, ACL 2023 - Toronto, Canada
Duration: 9 Jul 202314 Jul 2023

Conference

Conference61st Annual Meeting of the Association for Computational Linguistics, ACL 2023
LandCanada
ByToronto
Periode09/07/202314/07/2023
SponsorBloomberg Engineering, et al., Google Research, Liveperson, Meta, Microsoft
SeriesProceedings of the Annual Meeting of the Association for Computational Linguistics
ISSN0736-587X

Bibliographical note

Publisher Copyright:
© 2023 Association for Computational Linguistics.

ID: 374650746