PEARL: Physical Environment based Augmented Reality Lenses for In-Situ Human Movement Analysis

Research output: Contribution to book/conference proceedings/anthology/reportConference contributionContributedpeer-review


This paper presents Pearl, a mixed-reality approach for the analysis of human movement data in situ. As the physical environment shapes human motion and behavior, the analysis of such motion can benefit from the direct inclusion of the environment in the analytical process. We present methods for exploring movement data in relation to surrounding regions of interest, such as objects, furniture, and architectural elements. We introduce concepts for selecting and filtering data through direct interaction with the environment, and a suite of visualizations for revealing aggregated and emergent spatial and temporal relations. More sophisticated analysis is supported through complex queries comprising multiple regions of interest. To illustrate the potential of Pearl, we developed an Augmented Reality-based prototype and conducted expert review sessions and scenario walkthroughs in a simulated exhibition. Our contribution lays the foundation for leveraging the physical environment in the in-situ analysis of movement data.


Original languageEnglish
Title of host publicationCHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
ISBN (electronic)9781450394215
Publication statusPublished - 19 Apr 2023

External IDs

dblp conf/chi/LuoYRSGMD23
Scopus 85158099177
Mendeley f6f4b04c-68c2-372d-9c39-75b6316a0741
ORCID /0000-0002-1312-1528/work/142246466
ORCID /0000-0002-8923-6284/work/142247085
ORCID /0000-0002-3671-1619/work/142248365
ORCID /0000-0002-2176-876X/work/151435450



  • physical referents, affordance, movement data analysis, augmented/mixed reality, Immersive Analytics, In-situ visualization