Counterfactually Augmented Data and Unintended Bias: The Case of Sexism and Hate Speech Detection
Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review
Documents
- Fulltext
Final published version, 177 KB, PDF document
Counterfactually Augmented Data (CAD) aims to improve out-of-domain generalizability, an indicator of model robustness. The improvement is credited to promoting core features of the construct over spurious artifacts that happen to correlate with it. Yet, over-relying on core features may lead to unintended model bias. Especially, construct-driven CAD-perturbations of core features-may induce models to ignore the context in which core features are used. Here, we test models for sexism and hate speech detection on challenging data: non-hateful and nonsexist usage of identity and gendered terms. On these hard cases, models trained on CAD, especially construct-driven CAD, show higher false positive rates than models trained on the original, unperturbed data. Using a diverse set of CAD-construct-driven and construct-agnostic-reduces such unintended bias.
Original language | English |
---|---|
Title of host publication | Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies |
Publisher | Association for Computational Linguistics (ACL) |
Publication date | 2022 |
Pages | 4716-4726 |
ISBN (Electronic) | 9781955917711 |
DOIs | |
Publication status | Published - 2022 |
Event | 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2022 - Seattle, United States Duration: 10 Jul 2022 → 15 Jul 2022 |
Conference
Conference | 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2022 |
---|---|
Land | United States |
By | Seattle |
Periode | 10/07/2022 → 15/07/2022 |
Sponsor | Amazon, Bloomberg, et al., Google Research, LIVE PERSON, Meta |
Bibliographical note
Publisher Copyright:
© 2022 Association for Computational Linguistics.
ID: 341054609