Online Mouse Behavior Detection by Historical Dependency and Typical Instances

Abstract

Mouse behavior analysis plays a pivotal role in the research of numerous neurodegenerative diseases. In this paper, we develop a novel online mouse behavior detection approach, which can recognize mice behaviors in real-time videos and pinpoint the initiation and cessation points of target behaviors. In this architecture, the Long Short-Term Representation Aggregator (LSTRA) employs a designed temporal attention mechanism and integrates temporal dilated convolutions for multi-scale historical dependencies, overcoming the challenge of sparse behavior information due to subtle and brief mouse behaviors. Typical Instance Extractor (TIE) extracts representative frames for each category to calculate category-specific representations, addressing mouse body deformation challenges. The sinkhorn divergence-based constraint in our loss function ensures output congruence of these two modules. Extensive experiments on our PDMB-BD dataset and the public CRIM13 dataset demonstrate our approach achieves superior performance over state-of-the-art approaches.