Skip to content
naulon
Not checkedNot checked for notices yet.

Research on Continuous Dynamic Gesture Recognition of Chinese Sign Language Based on Multi-Mode Fusion

Jinquan Li, Jiaojiao Meng, Haijun Gong and 1 more

IEEE Access · 2022

doi:10.1109/access.2022.3212064 (opens in a new tab)

We email you when a new notice or a question about this paper appears.

Abstract

To solve the problem of the low recognition rate of continuous dynamic gestures in Chinese sign language, a non-invasive end-to-end continuous dynamic gesture recognition system combining Inertial Measurement Unit (IMU) signal and surface electromyography (sEMG) signal is constructed. After the preprocessing and fusion of the IMU signal and sEMG signal, the fusion gesture features which include acceleration, angular velocity, attitude quaternion, and sEMG information are extracted. A network model based on the Bi-directional Long-Short Term Memory (BiLSTM) network and Connectionist Temporal Classification (CTC) as loss function is then constructed, which avoids the adverse effects of inaccurate pre-segmentation of gesture sequences on continuous dynamic gesture recognition and realizes the end-to-end continuous dynamic gesture recognition. Finally, the Chinese Sign Language Database (CSLD), containing 10 kinds of Chinese sign language and 20000 gesture samples, is established. The training and testing of the database indicate that the average recognition rate reaches 98.66%, confirming the superiority of this method.

Abstract licence: CC BY-NC-ND

Reads

Read by people on this page

None yet.

Checked or read by AI through naulon

None yet.

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.

Are you an author?

Is this your paper?

Claim it to add your note and be paid when agents cite it. Free.