Pose Attention-based Knowledge Distillation for Human Action Recognition

초록

Human action recognition is a representative task in video understanding, which involves modeling the motion information of humans within a video. This motion information is closely related to the human pose estimation. In this work, we propose a multi-task framework for human action recognition by distilling knowledge related to motion information from human 2D pose estimation. To establish the combination of two distinct tasks, we propose a new network architecture that distills knowledge through the activation differences between a main network, which utilizes RGB frame sequences as input, and auxiliary network, which utilizes heatmap sequences generated by human 2D pose estimation as input. We show that the knowledge distilled through human 2D pose estimation leads to the attention of action-salient regions in videos, thereby improving the action recognition performance.