Higher-order Comparisons of Sentence Encoder Representations
Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review
Documents
- OA.Higher-order Comparisons of Sentence Encoder Representations
Final published version, 1.12 MB, PDF document
Representational Similarity Analysis (RSA) is a technique developed by neuroscientists for comparing activity patterns of different measurement modalities (e.g., fMRI, electrophysiology, behavior). As a framework, RSA has several advantages over existing approaches to interpretation of language encoders based on probing or diagnostic classification: namely, it does not require large training samples, is not prone to overfitting, and it enables a more transparent comparison between the representational geometries of different models and modalities. We demonstrate the utility of RSA by establishing a previously unknown correspondence between widely-employed pretrained language encoders and human processing difficulty via eye-tracking data, showcasing its potential in the interpretability toolbox for neural models.
Original language | English |
---|---|
Title of host publication | Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing |
Publisher | Association for Computational Linguistics |
Publication date | 2019 |
Pages | 5838–5845 |
Publication status | Published - 2019 |
Event | 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) - Hong Kong, China Duration: 3 Nov 2019 → 7 Nov 2019 |
Conference
Conference | 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) |
---|---|
Land | China |
By | Hong Kong |
Periode | 03/11/2019 → 07/11/2019 |
Number of downloads are based on statistics from Google Scholar and www.ku.dk
No data available
ID: 240321267