Anticipating Human Hand Grasps for Enhanced Interaction: A Bayesian Approach to Negative Latency in Human-Machine Interfaces

Research output: Contribution to journalResearch articleContributedpeer-review

Abstract

Human–machine interfaces increasingly rely on hand gesture recognition to enable intuitive interaction in immersive technologies and robotic systems. However, the inherent latency of gesture recognition pipelines, exacerbated by computational and network delays, remains a major obstacle to seamless real-time interaction. In this work, we introduce a lightweight, Bayesian model that leverages hand kinematics to predict target gestures before their completion, thereby recognizing hand gestures before they are completed (i.e. negative latency). Our approach is based on the continuous updating of posterior beliefs over a set of internal models, each with its gesture-specific dynamical system. These posteriors represent the probability that a gesture will be performed. We validate the model using a novel dataset of reach-tograsp tasks recorded with a sensor glove. Our results show that our model consistently predicts gestures several hundred milliseconds prior to completion. Furthermore, we show that reliable gesture prediction is achievable with reduced sensor dimensionality. These findings illustrate how anticipatory computing can substantially improve responsiveness and naturalness in a range of latency-sensitive applications, including immersive environments and robotic systems.

Details

Original languageEnglish
JournalIEEE access
Publication statusE-pub ahead of print - May 2026
Peer-reviewedYes

External IDs

ORCID /0000-0001-6870-5224/work/215831983
ORCID /0000-0001-8469-9573/work/215832040
ORCID /0000-0002-4590-1908/work/215835483

Keywords

Keywords

  • Computational neuroscience, Gesture recognition, Human-computer interaction, Intent recognition