Experiments with crowdsourced re-annotation of a POS tagging data set
Publikation: Bidrag til bog/antologi/rapport › Konferencebidrag i proceedings › Forskning › fagfællebedømt
Standard
Experiments with crowdsourced re-annotation of a POS tagging data set. / Hovy, Dirk; Plank, Barbara; Søgaard, Anders.
Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Baltimore, Maryland : Association for Computational Linguistics, 2014. s. 377-382.Publikation: Bidrag til bog/antologi/rapport › Konferencebidrag i proceedings › Forskning › fagfællebedømt
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - GEN
T1 - Experiments with crowdsourced re-annotation of a POS tagging data set
AU - Hovy, Dirk
AU - Plank, Barbara
AU - Søgaard, Anders
PY - 2014/6
Y1 - 2014/6
N2 - Crowdsourcing lets us collect multiple annotations for an item from several annotators. Typically, these are annotations for non-sequential classification tasks. While there has been some work on crowdsourcing named entity annotations, researchers have assumed that syntactic tasks such as part-of-speech (POS) tagging cannot be crowdsourced. This paper shows that workers can actually annotate sequential data almost as well as experts. Further, we show that the models learned from crowdsourced annotations fare as well as the models learned from expert annotations in downstream tasks.
AB - Crowdsourcing lets us collect multiple annotations for an item from several annotators. Typically, these are annotations for non-sequential classification tasks. While there has been some work on crowdsourcing named entity annotations, researchers have assumed that syntactic tasks such as part-of-speech (POS) tagging cannot be crowdsourced. This paper shows that workers can actually annotate sequential data almost as well as experts. Further, we show that the models learned from crowdsourced annotations fare as well as the models learned from expert annotations in downstream tasks.
M3 - Article in proceedings
SP - 377
EP - 382
BT - Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
PB - Association for Computational Linguistics
CY - Baltimore, Maryland
ER -
ID: 107673017