Around Device Interaction
Gesture interaction in the space around small devices is one way of overcoming limitations of touchscreen input. Gestures performed near a device allow users to interact more casually, for example: quickly dismissing incoming phone calls when busy; or checking for new text messages without having to pick the phone up. This style of interaction also allows input when touch is not suitable, for example: when hands are wet and messy from preparing food but you need to quickly check a recipe; or when wearing gloves outside in the cold. Finally, around-device interaction can also avoid problems caused by finger occlusion, where the interacting fingers cover parts of the display when providing touchscreen input. Wearable devices, like smart-watches, would especially benefit from less occlusion as screen sizes are much smaller than on phones.
Gesture Feedback
Good feedback is important to help users gesture effectively, although giving good feedback from small devices can be difficult. Visual feedback is not always ideal because it takes away already limited screen space. On-screen cues may also be difficult to notice from a short distance away. Instead, we look at how other output types can be used to give useful feedback. Audio and tactile feedback in particular are promising output modalities for gesture feedback.
Audio and tactile feedback are not without their own difficulties, however, and our research seeks to address these. One of the challenges of trying to deliver tactile feedback in a gesture interface is that users are not always in contact with the device they are interacting with; instead, they may be gesturing from several feet away or from across the room. Non-contact forms of tactile feedback (such as ultrasound haptics [?]) and distal methods of tactile feedback (such as wearable devices) should also be considered. We looked at these types of tactile feedback in our ICMI 2014 paper [?].
Gesture Design
We’ve also looked at gesture design for around-device interfaces [?]. In our paper we give recommendations for designing usable and acceptable mid-air gestures for mobile phones. We also present a set of evaluated gestures. In that work we also looked at the social acceptability of gestures, with similar findings to our earlier work on gesture acceptability [?].