Learning a POS tagger for AAVE-like language
Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review
POS taggers trained on newswire perform much worse on domains such as subtitles, lyrics, and tweets. In addition, these domains are very heterogeneous, and it is not clear what data to annotate to learn a POS tagger for subtitles, for example. In this paper we consider the problem of learning a POS tagger for subtitles, lyrics, and tweets associated with African-American Vernacular English from a previously released and manually annotated Twitter corpus. Our approach is to learn from a mixture of this data and unlabeled data, which was automatically and partially labeled using mined tag dictionaries. Our POS tagger obtains a tagging accuracy of 89% on subtitles, 85% on lyrics, and 83% on tweets, with up to 55% error reductions over a state-of-the-art newswire POS tagger, and 15-25% error reductions over a state-of-the-art Twitter POS tagger.
Original language | English |
---|---|
Title of host publication | Proceedings of NAACL-HLT 2016 |
Number of pages | 6 |
Publisher | Association for Computational Linguistics |
Publication date | 2016 |
Pages | 1115-1120 |
ISBN (Electronic) | ISBN 978-1-941643-91-4 |
Publication status | Published - 2016 |
Event | NAACL - San Diego, San Diego, United States Duration: 12 Jun 2016 → 17 Jun 2016 |
Conference
Conference | NAACL |
---|---|
Location | San Diego |
Land | United States |
By | San Diego |
Periode | 12/06/2016 → 17/06/2016 |
Links
- http://www.aclweb.org/anthology/N16-1130
Final published version
ID: 167551969