Syntactic Interchangeability in Word Embedding Models
Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review
Standard
Syntactic Interchangeability in Word Embedding Models. / Hershcovich, Daniel; Toledo, Assaf; Halfon, Alon; Slonim, Noam.
Proceedings of the 3rd Workshop on Evaluating Vector Space Representations for NLP. Association for Computational Linguistics, 2019. p. 70-76.Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - GEN
T1 - Syntactic Interchangeability in Word Embedding Models
AU - Hershcovich, Daniel
AU - Toledo, Assaf
AU - Halfon, Alon
AU - Slonim, Noam
PY - 2019
Y1 - 2019
N2 - Nearest neighbors in word embedding models are commonly observed to be semantically similar, but the relations between them can vary greatly. We investigate the extent to which word embedding models preserve syntactic interchangeability, as reflected by distances between word vectors, and the effect of hyper-parameters—context window size in particular. We use part of speech (POS) as a proxy for syntactic interchangeability, as generally speaking, words with the same POS are syntactically valid in the same contexts. We also investigate the relationship between interchangeability and similarity as judged by commonly-used word similarity benchmarks, and correlate the result with the performance of word embedding models on these benchmarks. Our results will inform future research and applications in the selection of word embedding model, suggesting a principle for an appropriate selection of the context window size parameter depending on the use-case.
AB - Nearest neighbors in word embedding models are commonly observed to be semantically similar, but the relations between them can vary greatly. We investigate the extent to which word embedding models preserve syntactic interchangeability, as reflected by distances between word vectors, and the effect of hyper-parameters—context window size in particular. We use part of speech (POS) as a proxy for syntactic interchangeability, as generally speaking, words with the same POS are syntactically valid in the same contexts. We also investigate the relationship between interchangeability and similarity as judged by commonly-used word similarity benchmarks, and correlate the result with the performance of word embedding models on these benchmarks. Our results will inform future research and applications in the selection of word embedding model, suggesting a principle for an appropriate selection of the context window size parameter depending on the use-case.
U2 - 10.18653/v1/W19-2009
DO - 10.18653/v1/W19-2009
M3 - Article in proceedings
SP - 70
EP - 76
BT - Proceedings of the 3rd Workshop on Evaluating Vector Space Representations for NLP
PB - Association for Computational Linguistics
T2 - Proceedings of the 3rd Workshop on Evaluating Vector Space Representations for
Y2 - 1 June 2019 through 1 June 2019
ER -
ID: 239016766