Syntactic Interchangeability in Word Embedding Models

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

Standard

Syntactic Interchangeability in Word Embedding Models. / Hershcovich, Daniel; Toledo, Assaf; Halfon, Alon; Slonim, Noam.

Proceedings of the 3rd Workshop on Evaluating Vector Space Representations for NLP. Association for Computational Linguistics, 2019. p. 70-76.

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

Harvard

Hershcovich, D, Toledo, A, Halfon, A & Slonim, N 2019, Syntactic Interchangeability in Word Embedding Models. in Proceedings of the 3rd Workshop on Evaluating Vector Space Representations for NLP. Association for Computational Linguistics, pp. 70-76, Proceedings of the 3rd Workshop on Evaluating Vector Space Representations for, Minneapolis, United States, 01/06/2019. https://doi.org/10.18653/v1/W19-2009

APA

Hershcovich, D., Toledo, A., Halfon, A., & Slonim, N. (2019). Syntactic Interchangeability in Word Embedding Models. In Proceedings of the 3rd Workshop on Evaluating Vector Space Representations for NLP (pp. 70-76). Association for Computational Linguistics. https://doi.org/10.18653/v1/W19-2009

Vancouver

Hershcovich D, Toledo A, Halfon A, Slonim N. Syntactic Interchangeability in Word Embedding Models. In Proceedings of the 3rd Workshop on Evaluating Vector Space Representations for NLP. Association for Computational Linguistics. 2019. p. 70-76 https://doi.org/10.18653/v1/W19-2009

Author

Hershcovich, Daniel ; Toledo, Assaf ; Halfon, Alon ; Slonim, Noam. / Syntactic Interchangeability in Word Embedding Models. Proceedings of the 3rd Workshop on Evaluating Vector Space Representations for NLP. Association for Computational Linguistics, 2019. pp. 70-76

Bibtex

@inproceedings{75f2180ded314616b3c856135dbd467c,
title = "Syntactic Interchangeability in Word Embedding Models",
abstract = "Nearest neighbors in word embedding models are commonly observed to be semantically similar, but the relations between them can vary greatly. We investigate the extent to which word embedding models preserve syntactic interchangeability, as reflected by distances between word vectors, and the effect of hyper-parameters—context window size in particular. We use part of speech (POS) as a proxy for syntactic interchangeability, as generally speaking, words with the same POS are syntactically valid in the same contexts. We also investigate the relationship between interchangeability and similarity as judged by commonly-used word similarity benchmarks, and correlate the result with the performance of word embedding models on these benchmarks. Our results will inform future research and applications in the selection of word embedding model, suggesting a principle for an appropriate selection of the context window size parameter depending on the use-case.",
author = "Daniel Hershcovich and Assaf Toledo and Alon Halfon and Noam Slonim",
year = "2019",
doi = "10.18653/v1/W19-2009",
language = "English",
pages = "70--76",
booktitle = "Proceedings of the 3rd Workshop on Evaluating Vector Space Representations for NLP",
publisher = "Association for Computational Linguistics",
note = "Proceedings of the 3rd Workshop on Evaluating Vector Space Representations for ; Conference date: 01-06-2019 Through 01-06-2019",

}

RIS

TY - GEN

T1 - Syntactic Interchangeability in Word Embedding Models

AU - Hershcovich, Daniel

AU - Toledo, Assaf

AU - Halfon, Alon

AU - Slonim, Noam

PY - 2019

Y1 - 2019

N2 - Nearest neighbors in word embedding models are commonly observed to be semantically similar, but the relations between them can vary greatly. We investigate the extent to which word embedding models preserve syntactic interchangeability, as reflected by distances between word vectors, and the effect of hyper-parameters—context window size in particular. We use part of speech (POS) as a proxy for syntactic interchangeability, as generally speaking, words with the same POS are syntactically valid in the same contexts. We also investigate the relationship between interchangeability and similarity as judged by commonly-used word similarity benchmarks, and correlate the result with the performance of word embedding models on these benchmarks. Our results will inform future research and applications in the selection of word embedding model, suggesting a principle for an appropriate selection of the context window size parameter depending on the use-case.

AB - Nearest neighbors in word embedding models are commonly observed to be semantically similar, but the relations between them can vary greatly. We investigate the extent to which word embedding models preserve syntactic interchangeability, as reflected by distances between word vectors, and the effect of hyper-parameters—context window size in particular. We use part of speech (POS) as a proxy for syntactic interchangeability, as generally speaking, words with the same POS are syntactically valid in the same contexts. We also investigate the relationship between interchangeability and similarity as judged by commonly-used word similarity benchmarks, and correlate the result with the performance of word embedding models on these benchmarks. Our results will inform future research and applications in the selection of word embedding model, suggesting a principle for an appropriate selection of the context window size parameter depending on the use-case.

U2 - 10.18653/v1/W19-2009

DO - 10.18653/v1/W19-2009

M3 - Article in proceedings

SP - 70

EP - 76

BT - Proceedings of the 3rd Workshop on Evaluating Vector Space Representations for NLP

PB - Association for Computational Linguistics

T2 - Proceedings of the 3rd Workshop on Evaluating Vector Space Representations for

Y2 - 1 June 2019 through 1 June 2019

ER -

ID: 239016766