TY - GEN

T1 - EntityBot

T2 - 2022 CHI Conference on Human Factors in Computing Systems, CHI EA 2022

AU - Vuong, Tung

AU - Andolina, Salvatore

AU - Jacucci, Giulio

AU - Daee, Pedram

AU - Klouche, Khalil

AU - Sjöberg, Mats

AU - Ruotsalo, Tuukka

AU - Kaski, Samuel

N1 - Publisher Copyright: © 2022 Owner/Author.

PY - 2022

Y1 - 2022

N2 - Our everyday digital tasks require access to information from a wide range of applications and systems. Although traditional search systems can help find information, they usually operate within one application (e.g., email client or web browser) and require the user's cognitive effort and attention to formulate proper search queries. In this paper, we demonstrate EntityBot, a system that proactively provides useful and supporting entities across application boundaries without requiring explicit query formulation. Our methodology is to exploit the context from screen frames captured every 2 seconds to recommend relevant entities for the current task. Recommendations are not restricted to only documents but include various kinds of entities, such as applications, documents, contact persons, and keywords representing the tasks. Recommendations are actionable, that is, a user can perform actions on recommended entities, such as opening documents and applications. The EntityBot also includes support for interactivity, allowing the user to affect the recommendations by providing explicit feedback on the entities. The usefulness of entity recommendations and their impact on user behavior has been evaluated in a user study based on real-world tasks. Quantitative and qualitative results suggest that the system had an actual impact on the tasks and led to high user satisfaction.

AB - Our everyday digital tasks require access to information from a wide range of applications and systems. Although traditional search systems can help find information, they usually operate within one application (e.g., email client or web browser) and require the user's cognitive effort and attention to formulate proper search queries. In this paper, we demonstrate EntityBot, a system that proactively provides useful and supporting entities across application boundaries without requiring explicit query formulation. Our methodology is to exploit the context from screen frames captured every 2 seconds to recommend relevant entities for the current task. Recommendations are not restricted to only documents but include various kinds of entities, such as applications, documents, contact persons, and keywords representing the tasks. Recommendations are actionable, that is, a user can perform actions on recommended entities, such as opening documents and applications. The EntityBot also includes support for interactivity, allowing the user to affect the recommendations by providing explicit feedback on the entities. The usefulness of entity recommendations and their impact on user behavior has been evaluated in a user study based on real-world tasks. Quantitative and qualitative results suggest that the system had an actual impact on the tasks and led to high user satisfaction.

KW - Proactive information retrieval

KW - real-world tasks

KW - user intent modeling

UR - http://www.scopus.com/inward/record.url?scp=85129765610&partnerID=8YFLogxK

U2 - 10.1145/3491101.3519910

DO - 10.1145/3491101.3519910

M3 - Article in proceedings

AN - SCOPUS:85129765610

T3 - Conference on Human Factors in Computing Systems - Proceedings

SP - 1

EP - 4

BT - CHI 2022 - Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems

PB - Association for Computing Machinery

Y2 - 30 April 2022 through 5 May 2022

ER -