The Role of Data Curation in Image Captioning
Publikation: Bidrag til bog/antologi/rapport › Konferencebidrag i proceedings › Forskning › fagfællebedømt
Dokumenter
- Fulltext
Forlagets udgivne version, 7,71 MB, PDF-dokument
Image captioning models are typically trained by treating all samples equally, neglecting to account for mismatched or otherwise difficult data points. In contrast, recent work has shown the effectiveness of training models by scheduling the data using curriculum learning strategies. This paper contributes to this direction by actively curating difficult samples in datasets without increasing the total number of samples. We explore the effect of using three data curation methods within the training process: complete removal of a sample, caption replacement, or image replacement via a text-to-image generation model. Experiments on the Flickr30K and COCO datasets with the BLIP and BEiT-3 models demonstrate that these curation methods do indeed yield improved image captioning models, underscoring their efficacy.
Originalsprog | Engelsk |
---|---|
Titel | EACL 2024 - 18th Conference of the European Chapter of the Association for Computational Linguistics, Proceedings of the Conference |
Redaktører | Yvette Graham, Matthew Purver, Matthew Purver |
Antal sider | 15 |
Forlag | Association for Computational Linguistics (ACL) |
Publikationsdato | 2024 |
Sider | 1074-1088 |
ISBN (Elektronisk) | 9798891760882 |
Status | Udgivet - 2024 |
Begivenhed | 18th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2024 - St. Julian's, Malta Varighed: 17 mar. 2024 → 22 mar. 2024 |
Konference
Konference | 18th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2024 |
---|---|
Land | Malta |
By | St. Julian's |
Periode | 17/03/2024 → 22/03/2024 |
Sponsor | Adobe, Babelscape, Bloomberg Engineering, Megagon Labs, Snowflake |
Bibliografisk note
Funding Information:
We thank Jiaang Li, Lei Li, and the CoAStal and LAMP groups for feedback. Wenyan Li is supported by the Lundbeck Foundation (Brain-Drugs grant: R279-2018-1145) and by Innovation Fund Denmark in the context of AI4Xray project. Jonas F. Lotz is funded by the ROCKWOOL Foundation (grant 1242). This work was supported by a research grant (VIL53122) from VILLUM FONDEN.
Publisher Copyright:
© 2024 Association for Computational Linguistics.
Links
- https://aclanthology.org/2024.eacl-long.65/
Forlagets udgivne version
ID: 392216501