PhD defence by Klemen Lilja

Klemen Lilja phd defence banner

Title

Hand-based Interaction in Mixed Reality: Manipulating, Navigating and Learning

Summary

Mixed reality has promised us better ways of interacting with physical and virtual worlds. Increased productivity, natural communication, on-demand user interfaces, immersive storytelling, and dream-like experiences.  Until now, few of these promises have come to fruition or been widely adopted. This is partly because of limitations of the current hardware and partly due to a lack of useful interaction techniques.  This thesis focuses on the latter.  I study mixed reality interaction techniques that help users manipulate objects, navigate virtual worlds, and support them while learning new skills.  The techniques are motivated by the wish to involve more of our bodies and expose the spatial capabilities of mixed reality.  Since hands are one of the most dexterous parts of our bodies, I based all the presented techniques on them.

The thesis investigates hand-based interaction techniques for mixed reality through four papers.  Paper 1 investigates how to best support free-hand manipulation of occluded objects with the help of augmented reality. Paper 2 explores re-configuration of virtual environments for enabling hand-based interactions at a distance. Paper 3 explores navigation of spatial recordings through direct manipulation of objects in the scene. Paper 4 investigates a technique for supporting motor skills acquisition through correction of users’ virtual hand movements. The four papers use a mix of quantitative and qualitative methods for evaluating the techniques. We conduct lab and remote experiments and use expert reviews and the think-aloud method. The tasks and applications are grounded in daily-life scenarios to make them understandable for a wide range of participants in our evaluations.

Combined, the papers present several hand-based interaction techniques for improving manipulation, navigation and learning in mixed reality. These techniques allow users to use greater capability of their hands and interact with the virtual and real world around them more fully.

Assessment Committee

  • Chair: Associate Professor, Daniel Ashbrook, Department of Computer Science
  • Professor, Stefania Serafin, Aalborg University
  • Associate Professor, Christian Holz, Eidgenössische Technische Hochschule Zürich, Switzerland

Academic supervisor

Professor, Kasper Hornbæk, Department of Computer Science, UCPH

 

For an electronic copy of the thesis, please go to:  https://di.ku.dk/english/research/phd/.