Byte-Level Grammatical Error Correction Using Synthetic and Curated Corpora
Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review
Standard
Byte-Level Grammatical Error Correction Using Synthetic and Curated Corpora. / Ingólfsdóttir, Svanhvít Lilja; Ragnarsson, Pétur Orri; Jónsson, Haukur Páll; Símonarson, Haukur Barri; Porsteinsson, Vilhjálmur; Snæbjarnarson, Vésteinn.
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics: Long Papers. Association for Computational Linguistics (ACL), 2023. p. 7299-7316.Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - GEN
T1 - Byte-Level Grammatical Error Correction Using Synthetic and Curated Corpora
AU - Ingólfsdóttir, Svanhvít Lilja
AU - Ragnarsson, Pétur Orri
AU - Jónsson, Haukur Páll
AU - Símonarson, Haukur Barri
AU - Porsteinsson, Vilhjálmur
AU - Snæbjarnarson, Vésteinn
N1 - Funding Information: We thank the Icelandic Language Technology Program (Nikulásdóttir et al., 2020). It has enabled the authors to focus on work in Icelandic NLP. Snæb-jarnarson was partially funded by the Pioneer Centre for AI, DNRF grant number P1, during the time of this work. Finally, we thank the anonymous reviewers for their helpful feedback. Funding Information: We thank the Icelandic Language Technology Program (Nikulásdóttir et al., 2020). It has enabled the authors to focus on work in Icelandic NLP. Snæbjarnarson was partially funded by the Pioneer Centre for AI, DNRF grant number P1, during the time of this work. Finally, we thank the anonymous reviewers for their helpful feedback. Publisher Copyright: © 2023 Association for Computational Linguistics.
PY - 2023
Y1 - 2023
N2 - Grammatical error correction (GEC) is the task of correcting typos, spelling, punctuation and grammatical issues in text. Approaching the problem as a sequence-to-sequence task, we compare the use of a common subword unit vocabulary and byte-level encoding. Initial synthetic training data is created using an error-generating pipeline, and used for finetuning two subword-level models and one byte-level model. Models are then finetuned further on hand-corrected error corpora, including texts written by children, university students, dyslexic and second-language writers, and evaluated over different error types and origins. We show that a byte-level model enables higher correction quality than a subword approach, not only for simple spelling errors, but also for more complex semantic, stylistic and grammatical issues. In particular, initial training on synthetic corpora followed by finetuning on a relatively small parallel corpus of real-world errors helps the byte-level model correct a wide range of commonly occurring errors. Our experiments are run for the Icelandic language but should hold for other similar languages, particularly morphologically rich ones.
AB - Grammatical error correction (GEC) is the task of correcting typos, spelling, punctuation and grammatical issues in text. Approaching the problem as a sequence-to-sequence task, we compare the use of a common subword unit vocabulary and byte-level encoding. Initial synthetic training data is created using an error-generating pipeline, and used for finetuning two subword-level models and one byte-level model. Models are then finetuned further on hand-corrected error corpora, including texts written by children, university students, dyslexic and second-language writers, and evaluated over different error types and origins. We show that a byte-level model enables higher correction quality than a subword approach, not only for simple spelling errors, but also for more complex semantic, stylistic and grammatical issues. In particular, initial training on synthetic corpora followed by finetuning on a relatively small parallel corpus of real-world errors helps the byte-level model correct a wide range of commonly occurring errors. Our experiments are run for the Icelandic language but should hold for other similar languages, particularly morphologically rich ones.
M3 - Article in proceedings
AN - SCOPUS:85174413901
SP - 7299
EP - 7316
BT - Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics
PB - Association for Computational Linguistics (ACL)
T2 - 61st Annual Meeting of the Association for Computational Linguistics, ACL 2023
Y2 - 9 July 2023 through 14 July 2023
ER -
ID: 371185212