Improving Spatial Interaction with Mobile Manipulation Robots Through Mixed Reality

Publikation: Beitrag in Buch/Konferenzbericht/Sammelband/Gutachten › Beitrag in Konferenzband › Beigetragen › Begutachtung

Abstract

Collaborative robots require intuitive interaction mechanisms that enable users to issue spatial commands and understand the robot’s behavior. However, traditional desktop interfaces rely on indirect two-dimensional representations of the robot’s three-dimensional workspace, requiring users to mentally translate between screen space and physical environment. This poses a particular problem for users without technical expertise. Mixed reality (MR) interfaces have the potential to address this limitation by enabling spatially grounded interaction through in-situ visualization and multimodal input. We present a multimodal MR interface that allows users to interact with a mobile manipulation robot using speech, gestures, and spatial visualizations of the robot’s internal state and intentions. We evaluate the interface in a within-subjects user study (N = 21) comparing MR interaction to two desktop-based interfaces with and without voice input. Results show that MR significantly reduces spatial command formulation time compared to both desktop interfaces. Perceived safety, control, usability, and workload ratings showed no significant omnibus differences, although equivalence for SUS and overall NASA-TLX was established only relative to the voice-enabled desktop baseline. These findings suggest that MR interfaces can improve the efficiency of spatial human-robot interaction, while broader claims about equivalent perceived experience require further study.

Details

OriginalspracheEnglisch
TitelVirtual Reality and Mixed Reality
Redakteure/-innenMario Lorenz, Gabriel Zachmann, Rachel Farrell, Voicu Popescu, Maki Sugimoto, Patrick Bourdot
Seiten284–308
Auflage1
ISBN (elektronisch)978-3-032-42847-9
PublikationsstatusVeröffentlicht - 1 Nov. 2026
Peer-Review-StatusJa

Publikationsreihe

ReiheLecture Notes in Computer Science
Band17151
ISSN0302-9743

Konferenz

Titel23rd EuroXR International Conference
KurztitelEuroXR 2026
Veranstaltungsnummer23
Dauer3 - 6 November 2026
Webseite
BekanntheitsgradInternationale Veranstaltung
OrtUniversitat Politècnica de València
StadtValencia
LandSpanien

Schlagworte

Schlagwörter

  • Mixed Reality, Human-Robot Interaction, Multimodal Interaction