Machine Translation of South African Sign Language

This project aims to build a machine translation system from South African Sign Language (SASL) to English text. Prior research has mainly focused on translation of English text to a sign language through the use of avatar systems. However, recent advances in technology in gesture recognition and animation can accurately capture and render facial, manual and body gestures.  Thus, the pieces required to build a fully-fledged translation system are in place, however synthesizing the output of gesture recognition into translatable linguistic text is needed.  This project focuses on creating an annotation schema suitable for text translation, as well as building a sign language to English bilingual corpus that can be used for machine learning of linguistic patterns. Creating an annotation schema, building and annotating a corpus requires linguistic knowledge, knowledge of gesture recognition systems, and sign language proficiency.

This project is a collaboration between the Computer Science and English Language & Linguistics departments at Rhodes. It builds on the Introduction to Sign Language Linguistics module taught in English Language and Linguistics 2 and collaborates with and builds on the gesture recognition and rendering work done by the Integration of Signed and Verbal Communication: South African Sign Language Recognition, Animation and Translation Group (SASL Group) at the University of the Western Cape.

This provides research opportunities for students in Linguistics as well as Computer Science. Scholarly impact will include further knowledge about SASL grammar on which there is very little published research. A bilingual SASL-English corpus will be developed for further research on SASL grammar and machine translation of sign languages in general.

Contact Ian Siebörger for more details at ian.sieborger@ru.ac.za