TapID: Rapid touch interaction in virtual reality using wearable sensing

M Meier, P Streli, A Fender… - 2021 IEEE Virtual Reality …, 2021 - ieeexplore.ieee.org
Current Virtual Reality systems typically use cameras to capture user input from controllers
or free-hand mid-air interaction. In this paper, we argue that this is a key impediment to …

Sensing beyond itself: Multi-functional use of ubiquitous signals towards wearable applications

Z Wang, J Li, Y **, J Wang, F Yang, G Li, X Ni… - Digital Signal …, 2021 - Elsevier
Wearable technologies provide a non-invasive way to monitor user's activity, identity, and
health in real-time, which have attracted tremendous interests from both academia and …

Echowrist: Continuous hand pose tracking and hand-object interaction recognition using low-power active acoustic sensing on a wristband

CJ Lee, R Zhang, D Agarwal, TC Yu, V Gunda… - Proceedings of the CHI …, 2024 - dl.acm.org
Our hands serve as a fundamental means of interaction with the world around us. Therefore,
understanding hand poses and interaction contexts is critical for human-computer …

Enabling voice-accompanying hand-to-face gesture recognition with cross-device sensing

Z Li, C Liang, Y Wang, Y Qin, C Yu, Y Yan… - Proceedings of the …, 2023 - dl.acm.org
Gestures performed accompanying the voice are essential for voice interaction to convey
complementary semantics for interaction purposes such as wake-up state and input …

SAWSense: Using surface acoustic waves for surface-bound event recognition

Y Iravantchi, Y Zhao, K Kin, AP Sample - … of the 2023 CHI Conference on …, 2023 - dl.acm.org
Enabling computing systems to understand user interactions with everyday surfaces and
objects can drive a wide range of applications. However, existing vibration-based sensors …

TouchInsight: Uncertainty-aware Rapid Touch and Text Input for Mixed Reality from Egocentric Vision

P Streli, M Richardson, F Botros, S Ma… - Proceedings of the 37th …, 2024 - dl.acm.org
While passive surfaces offer numerous benefits for interaction in mixed reality, reliably
detecting touch input solely from head-mounted cameras has been a long-standing …

Sensurfaces: A novel approach for embedded touch sensing on everyday surfaces

B Parilusyan, M Teyssier, V Martinez-Missir… - Proceedings of the …, 2022 - dl.acm.org
Ubiquitous touch sensing surfaces are largely influenced by touchscreens' look and feel and
fail to express the physical richness of existing surrounding materials. We introduce …

MAF: Exploring Mobile Acoustic Field for Hand-to-Face Gesture Interactions

Y Yang, T Chen, Y Huang, X Guo… - Proceedings of the CHI …, 2024 - dl.acm.org
We present MAF, a novel acoustic sensing approach that leverages the commodity
hardware in bone conduction earphones for hand-to-face gesture interactions. Briefly, by …

EgoPressure: A Dataset for Hand Pressure and Pose Estimation in Egocentric Vision

Y Zhao, T Kwon, P Streli, M Pollefeys… - arxiv preprint arxiv …, 2024 - arxiv.org
Estimating touch contact and pressure in egocentric vision is a central task for downstream
applications in Augmented Reality, Virtual Reality, as well as many robotic applications …

Structured Light Speckle: Joint Ego-Centric Depth Estimation and Low-Latency Contact Detection via Remote Vibrometry

P Streli, J Jiang, J Rossie, C Holz - Proceedings of the 36th Annual ACM …, 2023 - dl.acm.org
Despite advancements in egocentric hand tracking using head-mounted cameras, contact
detection with real-world objects remains challenging, particularly for the quick motions often …