Action Detection for Wildlife Monitoring with Camera Traps Based on Segmentation with Filtering of Tracklets (SWIFT) and Mask-Guided Action Recognition (MAROON)

Publikation: Beitrag in FachzeitschriftForschungsartikelBeigetragenBegutachtung

Beitragende

  • Frank Schindler - , Universität Bonn (Autor:in)
  • Volker Steinhage - , Universität Bonn (Autor:in)
  • Suzanne T.S. van Beeck Calkoen - , Professur für Forstzoologie, Bavarian Forest National Park, Georg-August-Universität Göttingen (Autor:in)
  • Marco Heurich - , Bavarian Forest National Park, Albert-Ludwigs-Universität Freiburg, Inland Norway University of Applied Sciences (Autor:in)

Abstract

Behavioral analysis of animals in the wild plays an important role for ecological research and conservation and has been mostly performed by researchers. We introduce an action detection approach that automates this process by detecting animals and performing action recognition on the detected animals in camera trap videos. Our action detection approach is based on SWIFT (segmentation with filtering of tracklets), which we have already shown to successfully detect and track animals in wildlife videos, and MAROON (mask-guided action recognition), an action recognition network that we are introducing here. The basic ideas of MAROON are the exploitation of the instance masks detected by SWIFT and a triple-stream network. The instance masks enable more accurate action recognition, especially if multiple animals appear in a video at the same time. The triple-stream approach extracts features for the motion and appearance of the animal. We evaluate the quality of our action recognition on two self-generated datasets, from an animal enclosure and from the wild. These datasets contain videos of red deer, fallow deer and roe deer, recorded both during the day and night. MAROON improves the action recognition accuracy compared to other state-of-the-art approaches by an average of 10 percentage points on all analyzed datasets and achieves an accuracy of (Formula presented.) on the Rolandseck Daylight dataset, in which 11 different action classes occur. Our action detection system makes it possible todrasticallyreduce the manual work of ecologists and at the same time gain new insights through standardized results.

Details

OriginalspracheEnglisch
Aufsatznummer514
Seitenumfang17
FachzeitschriftApplied Sciences (Switzerland)
Jahrgang14 (2024)
Ausgabenummer2
PublikationsstatusVeröffentlicht - 6 Jan. 2024
Peer-Review-StatusJa

Schlagworte

Schlagwörter

  • action detection for deer, deep learning, mask-supported action recognition, triple-stream convolutional neural network, video instance segmentation, wildlife monitoring