A novel approach for automatic annotation of human actions in 3D point clouds for flexible collaborative tasks with industrial robots

Research output: Contribution to journalResearch articleContributedpeer-review

Contributors

  • Sebastian Krusche - , Fraunhofer Institute for Machine Tools and Forming Technology (Author)
  • Ibrahim Al Naser - , Fraunhofer Institute for Machine Tools and Forming Technology (Author)
  • Mohamad Bdiwi - , Fraunhofer Institute for Machine Tools and Forming Technology (Author)
  • Steffen Ihlenfeldt - , Chair of Machine Tools Development and Adaptive Controls, Fraunhofer Institute for Machine Tools and Forming Technology (Author)

Abstract

Manual annotation for human action recognition with content semantics using 3D Point Cloud (3D-PC) in industrial environments consumes a lot of time and resources. This work aims to recognize, analyze, and model human actions to develop a framework for automatically extracting content semantics. Main Contributions of this work: 1. design a multi-layer structure of various DNN classifiers to detect and extract humans and dynamic objects using 3D-PC preciously, 2. empirical experiments with over 10 subjects for collecting datasets of human actions and activities in one industrial setting, 3. development of an intuitive GUI to verify human actions and its interaction activities with the environment, 4. design and implement a methodology for automatic sequence matching of human actions in 3D-PC. All these procedures are merged in the proposed framework and evaluated in one industrial Use-Case with flexible patch sizes. Comparing the new approach with standard methods has shown that the annotation process can be accelerated by 5.2 times through automation.

Details

Original languageEnglish
Article number1028329
JournalFrontiers in robotics and AI
Volume10
Publication statusPublished - 2023
Peer-reviewedYes

Keywords

Keywords

  • data labeling, deep learning, human activity recognition, point cloud annotation, robotics