Recognition System of Indonesia Sign Language based on Sensor and Artificial Neural Network

Authors

Endang Supriyati, Informatics Engineering Department, Muria Kudus University, Gondangmanis Kudus 59352, IndonesiaFollow
Mohammad Iqbal, Electrical Engineering Department, Muria Kudus University, Gondangmanis Kudus 59352, Indonesia

Abstract

Sign language as a kind of gestures is one of the most natural ways of communication for most people in deaf community. The aim of the sign language recognition is to provide a translation for sign gestures into meaningful text or speech so that communication between deaf and hearing society can easily be made. In this research, the Indonesian sign language recognition system based on flex sensors and an accelerometer is developed. This recognition system uses a sensory glove to capture data. The sensor data that are processed into feature vector are the 5-fingers bending and the palm acceleration when performing the sign language. The most important part of the recognition system is a feature extraction. In this research, histogram is used as feature extraction. The extracted features are used as data training and data testing for Adaptive Neighborhood based Modified Backpropagation (ANMBP). The system is implemented and tested using a data set of 1000 samples of 50 Indonesia sign, 20 samples for each sign. Among these 500 data were used as the training data, and the remaining 500 data were used as the testing data. The system obtains the recognition rate of 91.60% in offline mode.

Bahasa Abstract

Sistem Pengenalan Bahasa Isyarat Indonesia Berbasis Sensor dan Jaringan Saraf Tiruan. Bahasa isyarat adalah salah satu cara yang paling alami dalam berkomunikasi bagi kaum tuna rungu. Tujuan pengenalan bahasa isyarat adalah untuk menerjemahkan bahasa isyarat ke dalam bentuk teks dan atau suara sehingga komunikasi antara kaum tuna rungu dengan masyarakat luas dapat terjalin. Penelitian ini mengembangkan sistem isyarat bahasa Indonesia berbasis sensor flex dan accelerometer. Sistem pengenalan ini menggunakan sarung tangan bersensor untuk mengumpulkan data. Data sensor yang diolah menjadi vektor ciri adalah data tekukan kelima jari tangan dan akselarasi telapak tangan selama melakukan gerak isyarat kata. Bagian terpenting pada sistem pengenalan adalah ekstraksi fitur. Proses ekstraksi fitur dalam penelitian ini menggunakan histogram. Hasil ekstraksi fitur digunakan sebagai data training dan data testing pada sistem pengenal menggunakan Adaptive Neighborhood based Modified Backpropagation (ANMBP). Sistem pengenalan bahasa isyarat ini diimplementasikan dan diuji coba dengan menggunakan 1000 data dari 50 gerak isyarat, dengan jumlah 20 data untuk tiap gerak isyarat. 500 data digunakan untuk data training dan 500 data untuk data testing. Hasil pengujian mendapatkan akurasi 91,60% dalam mode offline.

References

1. Q. Munib, M. Habeeb, B. Takruri, H.A Al-Malik, Expert Sistems Appl. 32 (2007) 24.
2. Y.-H. Lee, C.-Y. Tsai, Expert Sistems Appl. 36 (2009) 1123.
3. M. AL-Rousan, K. Assaleh, A. Tala’a, Appl. Soft Comput. 9 (2009) 990.
4. W. Gao, G. Fanga, D. Zhaoa, Y. Chen, Pattern Recognit. 37 (2004) 2389.
5. C. Oz, M.C. Leu, Neuro Comput. 70 (2007) 2891.
6. M.H. Yang, N. Ahuja, IEEE Conference on Computer Vision and Pattern Recognition, Fort Collins, Colorado, 1 (1999) 466.
7. T. Kathirvalakumar, S.J. Subavathi, Neuro Comput. 72 (2009) 3915.
8. E.T. Sekar, Tesis Magister ITB, Bandung, Indonesia, 2001.
9. Spectrasymbol, Flex Sensor, http://spectrasymbol.com, 2012.
10. Parallax Inc., Hitachi® H48C 3-Axis Accelerometer Module, http://www.parallax.com, 2007.
11. S. Abid, F. Fnaiech, M. Najim, IEEE Trans. Neural Netw. 12 (2001) 424.
12. A. Toledo, M. Pinzolas, J.J. Ibarrola, G. Lera, IEEE Trans. Neural Netw. 16 (2005) 988.
13. L. Behera, S. Kumar, A. Patnaik, IEEE Trans. Neural Netw. 17 (2006) 1116.
14. PT. Telekomunikasi Indonesia, Tbk., i-CHAT, http://app.i-chat.web.id, 2010.
15. H.P. Mauridhi, K. Agus, Supervised Neural Network dan Aplikasinya, Penerbit Graha Ilmu, Yogyakarta, 2006, p.176.
16. A. Akl, Thesis, Department of Electrical and Computer Engineering, University of Toronto, 2010.
17. J. Liua, L. Zhong, J. Wickramasuriya, V. Vasudevan, Pervasive and Mobile Comput. 5 (2009) 657.
18. R. Kohavi, F. Provost, Glossary of Terms, Mach. Learn. 30/2-3 (1998) 271.
19. M.M. Fernandez, Fundamentals on Estimation Theory, http://lmi.bwh.harvard.edu/papers/pdfs/2004/martin-fernandezCOURSE04b.pdf, access on 5-Peb-2010.

Recommended Citation

Supriyati, Endang and Iqbal, Mohammad (2013) "Recognition System of Indonesia Sign Language based on Sensor and Artificial Neural Network," Makara Journal of Technology: Vol. 17: Iss. 1, Article 6.
DOI: 10.7454/mst.v17i1.1924
Available at: https://scholarhub.ui.ac.id/mjt/vol17/iss1/6