Bionic Tracking: Using Eye Tracking to Track Biological Cells in Virtual Reality

Research output: Contribution to book/Conference proceedings/Anthology/ReportConference contributionContributedpeer-review

Contributors

Abstract

We present Bionic Tracking, a novel method for solving biological cell tracking problems with eye tracking in virtual reality using commodity hardware. Using gaze data, and especially smooth pursuit eye movements, we are able to track cells in time series of 3D volumetric datasets. The problem of tracking cells is ubiquitous in developmental biology, where large volumetric microscopy datasets are acquired on a daily basis, often comprising hundreds or thousands of time points that span hours or days. The image data, however, is only a means to an end, and scientists are often interested in the reconstruction of cell trajectories and cell lineage trees. Reliably tracking cells in crowded three-dimensional space over many time points remains an open problem, and many current approaches rely on tedious manual annotation or curation. In the Bionic Tracking approach, we substitute the usual 2D point-and-click interface for annotation or curation with eye tracking in a virtual reality headset, where users follow cells with their eyes in 3D space in order to track them. We detail the interaction design of our approach and explain the graph-based algorithm used to connect different time points, also taking occlusion and user distraction into account. We demonstrate Bionic Tracking using examples from two different biological datasets. Finally, we report on a user study with seven cell tracking experts, highlighting the benefits and limitations of Bionic Tracking compared to point-and-click interfaces.

Details

Original languageEnglish
Title of host publicationComputer Vision – ECCV 2020 Workshops
EditorsAdrien Bartoli, Andrea Fusiello
PublisherSpringer, Berlin [u. a.]
Pages280-297
Number of pages18
ISBN (print)9783030664145
Publication statusPublished - 2020
Peer-reviewedYes

Publication series

SeriesLecture Notes in Computer Science, Volume 12535
ISSN0302-9743

Conference

TitleWorkshops held at the 16th European Conference on Computer Vision, ECCV 2020
Duration23 - 28 August 2020
CityGlasgow
CountryUnited Kingdom

External IDs

ORCID /0000-0003-4414-4340/work/142252160
ORCID /0000-0002-2176-876X/work/151435416

Keywords