A Two-channel Attention Mechanism-based MobileNetV2 And Bidirectional Long Short Memory Network For Multi-modal Dimension Dance Emotion Recognition
Shuang Gao
Journal of Applied Science and Engineering · 2022
doi:10.6180/jase.202304_26(4).0001 (opens in a new tab)
We email you when a new notice or a question about this paper appears.
Abstract
In recent years, dance emotion recognition has become a hot research issue in the field of human-computer interaction and art design. However, multi-modal dimension emotion recognition can detect subtle action emotion changes. In multi-modal dimension emotion recognition, it is necessary to consider how to effectively fuse different modal emotion information. Aiming at the problem of effective feature extraction and modal synchronization in feature layer fusion, and the correlation problem of different modal feature information in decision layer fusion, we propose a new method for multi-modal dimension dance emotion recognition. It takes the lightweight network structure MobileNetV2 as the benchmark network architecture and introduces an independently designed attention module to increase the weight of the saliency feature graph learned by the convolutional layer, so as to improve the classification accuracy. The spatial information of the proposed model is input in parallel with the channel information. By defining the small convolution of 1×1 and 3×3, the computational amount and complexity are reduced while ensuring the lightweight of the structure. The audio model, video model and multi-modal fusion model are respectively constructed to carry out deep feature learning for information flow. Finally, they are input to the two-channel bi-directional long short term memory to obtain the final motion emotion prediction value. Compared with other state-of-the-art methods, the proposed method can effectively capture emotion information in the high-level dimension and obtain the better recognition effect, so it can better fuse audio and video information.
Abstract licence: CC BY
Reads
Read by people on this page
None yet.
Checked or read by AI through naulon
None yet.
A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.
Are you an author?
Is this your paper?
Claim it to add your note and be paid when agents cite it. Free.