Music2Body: LSTM 기반 댄스 모션 생성 방법

초록

Automatic human movement generation is a challenging computational problem. It becomes even more strenuous in the case of choreography motion. In this work, we present a lightweight LSTM-based network architecture for generating choreography. First, we create a dataset with dance cover videos from YouTube. Next, we extract human body joint points using state-of-the-art 3D pose estimator and reduce the dimensionality of these points using Principal Component Analysis (PCA). To preprocess audio, we use Mel-Frequency Cepstrum Coefficients (MFCC), which proved to be effective in various music classification tasks. Then we use the obtained MFCC and PCA coefficients for training and testing of our model. The experimental results demonstrate that despite being able to generate plausible dance movements for a variety of songs, the proposed model suits best for K-pop genre.