NLP tasks are often limited by scarcity of
manually annotated data. In social media
sentiment analysis and related tasks,
researchers have therefore used binarized
emoticons and specific hashtags as forms
of distant supervision. Our paper shows
that by extending the distant supervision
to a more diverse set of noisy labels, the
models can learn richer representations.
Through emoji prediction on a dataset of
1246 million tweets containing one of 64
common emojis we obtain state-of-theart
performance on 8 benchmark datasets
within emotion, sentiment and sarcasm detection
using a single pretrained model.
Our analyses confirm that the diversity of
our emotional labels yield a performance
improvement over previous distant supervision
approaches.