Learning Geometric Representations of Objects via Interaction
Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review
Documents
- Fulltext
Submitted manuscript, 6.49 MB, PDF document
We address the problem of learning representations from observations of a scene involving an agent and an external object the agent interacts with. To this end, we propose a representation learning framework extracting the location in physical space of both the agent and the object from unstructured observations of arbitrary nature. Our framework relies on the actions performed by the agent as the only source of supervision, while assuming that the object is displaced by the agent via unknown dynamics. We provide a theoretical foundation and formally prove that an ideal learner is guaranteed to infer an isometric representation, disentangling the agent from the object and correctly extracting their locations. We evaluate empirically our framework on a variety of scenarios, showing that it outperforms vision-based approaches such as a state-of-the-art keypoint extractor. We moreover demonstrate how the extracted representations enable the agent to solve downstream tasks via reinforcement learning in an efficient manner.
Original language | English |
---|---|
Title of host publication | Machine Learning and Knowledge Discovery in Databases : Research Track - European Conference, ECML PKDD 2023, Proceedings |
Editors | Danai Koutra, Claudia Plant, Manuel Gomez Rodriguez, Elena Baralis, Francesco Bonchi |
Volume | 4 |
Publisher | Springer |
Publication date | 2023 |
Pages | 629-644 |
ISBN (Print) | 9783031434204 |
DOIs | |
Publication status | Published - 2023 |
Event | European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2023 - Turin, Italy Duration: 18 Sep 2023 → 22 Sep 2023 |
Conference
Conference | European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2023 |
---|---|
Land | Italy |
By | Turin |
Periode | 18/09/2023 → 22/09/2023 |
Series | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
---|---|
Volume | 14172 LNAI |
ISSN | 0302-9743 |
Bibliographical note
Funding Information:
Acknowledgements. This work was supported by the Swedish Research Council, the Knut and Alice Wallenberg Foundation, the European Research Council (ERC-BIRD-884807) and the European Horizon 2020 CANOPIES project. Hang Yin would like to acknolwedge the support by the Pioneer Centre for AI, DNRF grant number P1.
Publisher Copyright:
© 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.
- Equivariance, Interaction, Representation Learning
Research areas
ID: 390400542