Pervasive Hand Gesture Recognition for Smartphones using Non-audible\n Sound and Deep Learning

Résumé

Due to the mass advancement in ubiquitous technologies nowadays, new\npervasive methods have come into the practice to provide new innovative\nfeatures and stimulate the research on new human-computer interactions. This\npaper presents a hand gesture recognition method that utilizes the smartphone's\nbuilt-in speakers and microphones. The proposed system emits an ultrasonic\nsonar-based signal (inaudible sound) from the smartphone's stereo speakers,\nwhich is then received by the smartphone's microphone and processed via a\nConvolutional Neural Network (CNN) for Hand Gesture Recognition. Data\naugmentation techniques are proposed to improve the detection accuracy and\nthree dual-channel input fusion methods are compared. The first method merges\nthe dual-channel audio as a single input spectrogram image. The second method\nadopts early fusion by concatenating the dual-channel spectrograms. The third\nmethod adopts late fusion by having two convectional input branches processing\neach of the dual-channel spectrograms and then the outputs are merged by the\nlast layers. Our experimental results demonstrate a promising detection\naccuracy for the six gestures presented in our publicly available dataset with\nan accuracy of 93.58\\% as a baseline.\n

Citer ce document

Accès au document

Ce document est hébergé sur son dépôt institutionnel d'origine.

Auteur(s)

Statistiques

Consultations : 1

Téléchargements : 0