Counterfactually Augmented Data and Unintended Bias: The Case of Sexism and Hate Speech Detection
Publikation: Bidrag til bog/antologi/rapport › Konferencebidrag i proceedings › Forskning › fagfællebedømt
Dokumenter
- Fulltext
Forlagets udgivne version, 177 KB, PDF-dokument
Counterfactually Augmented Data (CAD) aims to improve out-of-domain generalizability, an indicator of model robustness. The improvement is credited to promoting core features of the construct over spurious artifacts that happen to correlate with it. Yet, over-relying on core features may lead to unintended model bias. Especially, construct-driven CAD-perturbations of core features-may induce models to ignore the context in which core features are used. Here, we test models for sexism and hate speech detection on challenging data: non-hateful and nonsexist usage of identity and gendered terms. On these hard cases, models trained on CAD, especially construct-driven CAD, show higher false positive rates than models trained on the original, unperturbed data. Using a diverse set of CAD-construct-driven and construct-agnostic-reduces such unintended bias.
Originalsprog | Engelsk |
---|---|
Titel | Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies |
Forlag | Association for Computational Linguistics (ACL) |
Publikationsdato | 2022 |
Sider | 4716-4726 |
ISBN (Elektronisk) | 9781955917711 |
DOI | |
Status | Udgivet - 2022 |
Begivenhed | 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2022 - Seattle, USA Varighed: 10 jul. 2022 → 15 jul. 2022 |
Konference
Konference | 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2022 |
---|---|
Land | USA |
By | Seattle |
Periode | 10/07/2022 → 15/07/2022 |
Sponsor | Amazon, Bloomberg, et al., Google Research, LIVE PERSON, Meta |
Bibliografisk note
Funding Information:
We thank the members of the Computational Social Science department at GESIS, the CopeNLU group, and the anonymous reviewers for their constructive feedback. Isabelle Augenstein’s research is partially funded by a DFF Sapere Aude research leader grant with grant number 0171-00034B.
Publisher Copyright:
© 2022 Association for Computational Linguistics.
ID: 341054609