Which One is Better? Self-supervised Temporal Coherence Learning for Skeleton Based Action Recognition

Bizhu Wu, Mingyan Wu, Haoqin Ji, Linlin Shen

Research output: Chapter in Book/Conference proceedingConference contributionpeer-review

Abstract

Recently, researchers have achieved significant results in the skeleton based action recognition task. To better model the skeleton sequences, existing methods learned the feature representations in the self-supervised setting by solving pretext tasks, such as predicting the order of a shuffled skeleton sequence or verifying whether a given skeleton sequence is shuffled or not. However, these pretext tasks are either too challenging or too easy for the encoder to obtain a proper skeleton representation for action recognition. Therefore, we propose a novel self-pretraining pretext task, Which One Is Better (WOIB), to identify which one is more temporally coherent, given two shuffled skeleton sequences. Experiments on the NTU RGB+D, NTU RGB+D 120, and Kinetics-Skeleton datasets with different network architectures show significant improvements in recognition accuracy, demonstrating that such a well-designed pretext task is general and able to drive the encoder to learn more discriminative representations.

Original languageEnglish
Title of host publication2022 IEEE International Joint Conference on Biometrics, IJCB 2022
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781665463942
DOIs
Publication statusPublished - 2022
Externally publishedYes
Event2022 IEEE International Joint Conference on Biometrics, IJCB 2022 - Abu Dhabi, United Arab Emirates
Duration: 10 Oct 202213 Oct 2022

Publication series

Name2022 IEEE International Joint Conference on Biometrics, IJCB 2022

Conference

Conference2022 IEEE International Joint Conference on Biometrics, IJCB 2022
Country/TerritoryUnited Arab Emirates
CityAbu Dhabi
Period10/10/2213/10/22

ASJC Scopus subject areas

  • Agricultural and Biological Sciences (miscellaneous)
  • Computer Vision and Pattern Recognition
  • Health Informatics
  • Instrumentation

Fingerprint

Dive into the research topics of 'Which One is Better? Self-supervised Temporal Coherence Learning for Skeleton Based Action Recognition'. Together they form a unique fingerprint.

Cite this