Improving Spatial Interaction with Mobile Manipulation Robots Through Mixed Reality
Research output: Contribution to book/Conference proceedings/Anthology/Report › Conference contribution › Contributed › peer-review
Contributors
Abstract
Collaborative robots require intuitive interaction mechanisms that enable users to issue spatial commands and understand the robot’s behavior. However, traditional desktop interfaces rely on indirect two-dimensional representations of the robot’s three-dimensional workspace, requiring users to mentally translate between screen space and physical environment. This poses a particular problem for users without technical expertise. Mixed reality (MR) interfaces have the potential to address this limitation by enabling spatially grounded interaction through in-situ visualization and multimodal input. We present a multimodal MR interface that allows users to interact with a mobile manipulation robot using speech, gestures, and spatial visualizations of the robot’s internal state and intentions. We evaluate the interface in a within-subjects user study (N = 21) comparing MR interaction to two desktop-based interfaces with and without voice input. Results show that MR significantly reduces spatial command formulation time compared to both desktop interfaces. Perceived safety, control, usability, and workload ratings showed no significant omnibus differences, although equivalence for SUS and overall NASA-TLX was established only relative to the voice-enabled desktop baseline. These findings suggest that MR interfaces can improve the efficiency of spatial human-robot interaction, while broader claims about equivalent perceived experience require further study.
Details
| Original language | English |
|---|---|
| Title of host publication | Virtual Reality and Mixed Reality |
| Editors | Mario Lorenz, Gabriel Zachmann, Rachel Farrell, Voicu Popescu, Maki Sugimoto, Patrick Bourdot |
| Pages | 284–308 |
| Edition | 1 |
| ISBN (electronic) | 978-3-032-42847-9 |
| Publication status | Published - 1 Nov 2026 |
| Peer-reviewed | Yes |
Publication series
| Series | Lecture Notes in Computer Science |
|---|---|
| Volume | 17151 |
| ISSN | 0302-9743 |
Conference
| Title | 23rd EuroXR International Conference |
|---|---|
| Abbreviated title | EuroXR 2026 |
| Conference number | 23 |
| Duration | 3 - 6 November 2026 |
| Website | |
| Degree of recognition | International event |
| Location | Universitat Politècnica de València |
| City | Valencia |
| Country | Spain |
Keywords
Keywords
- Mixed Reality, Human-Robot Interaction, Multimodal Interaction