Neural naturalist: Generating fine-grained image comparisons
Research output: Contribution to journal › Conference article › Research › peer-review
Standard
Neural naturalist : Generating fine-grained image comparisons. / Forbes, Maxwell; Kaeser-Chen, Christine; Sharma, Piyush; Belongie, Serge.
In: EMNLP-IJCNLP 2019 - 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing, Proceedings of the Conference, 2020, p. 708-717.Research output: Contribution to journal › Conference article › Research › peer-review
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - GEN
T1 - Neural naturalist
T2 - 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019
AU - Forbes, Maxwell
AU - Kaeser-Chen, Christine
AU - Sharma, Piyush
AU - Belongie, Serge
N1 - Publisher Copyright: © 2019 Association for Computational Linguistics
PY - 2020
Y1 - 2020
N2 - We introduce the new Birds-to-Words dataset of 41k sentences describing fine-grained differences between photographs of birds. The language collected is highly detailed, while remaining understandable to the everyday observer (e.g., “heart-shaped face,” “squat body”). Paragraph-length descriptions naturally adapt to varying levels of taxonomic and visual distance-drawn from a novel stratified sampling approach-with the appropriate level of detail. We propose a new model called Neural Naturalist that uses a joint image encoding and comparative module to generate comparative language, and evaluate the results with humans who must use the descriptions to distinguish real images. Our results indicate promising potential for neural models to explain differences in visual embedding space using natural language, as well as a concrete path for machine learning to aid citizen scientists in their effort to preserve biodiversity.
AB - We introduce the new Birds-to-Words dataset of 41k sentences describing fine-grained differences between photographs of birds. The language collected is highly detailed, while remaining understandable to the everyday observer (e.g., “heart-shaped face,” “squat body”). Paragraph-length descriptions naturally adapt to varying levels of taxonomic and visual distance-drawn from a novel stratified sampling approach-with the appropriate level of detail. We propose a new model called Neural Naturalist that uses a joint image encoding and comparative module to generate comparative language, and evaluate the results with humans who must use the descriptions to distinguish real images. Our results indicate promising potential for neural models to explain differences in visual embedding space using natural language, as well as a concrete path for machine learning to aid citizen scientists in their effort to preserve biodiversity.
UR - http://www.scopus.com/inward/record.url?scp=85084291583&partnerID=8YFLogxK
M3 - Conference article
AN - SCOPUS:85084291583
SP - 708
EP - 717
JO - EMNLP-IJCNLP 2019 - 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing, Proceedings of the Conference
JF - EMNLP-IJCNLP 2019 - 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing, Proceedings of the Conference
Y2 - 3 November 2019 through 7 November 2019
ER -
ID: 301823194