Combining multi-touch and mid-air gestures

Master thesis defence by Marc Framvig Antonsen and Juri Rosenkilde


This thesis describes how multi-touch and mid-air interaction can be combined to improve the interaction experience on wall-sized displays.

To combine touch and mid-air, we developed a tracker that can track user movements in mid air. With the tracker, we implemented body- gesture recognition and mid-air pointing. We then designed several proposals for interaction methods that combine touch and mid-air interaction. We focused further on two of these proposals, both using mid-air pointing, and evaluated them in an empirical point and click study against regular touch and a gyroscopic mouse. The experiments showed that our methods performed worse than the control methods, but scored well on user satisfaction. On the basis of the experiment, we argue that further improvement of the tracker is necessary, and recommends investigating the methods with drag and drop tasks.

Master thesis advisor:

Kasper Hornbæk

External examiner:

Mikael Skov, Aalborg University