Auslan Communication Technology Pipeline
The Auslan Communication Technologies Pipeline project is a three-year project which will develop a modular pipeline of artificial intelligence systems. It is part of the Trusted Autonomous Systems Defence CRC.
TAS-DCRC Webinar 29 Sep 2020
The Recognition System will be a machine learning model trained to recognise conversational Auslan. The Production System will generate artificial yet human-like Auslan signing. The Processing System will take the input from the Recognition System, process it, and generate output through the Production System. The exact form of the Recognition System will be decided in consultation with members of Australia’s Deaf community, but could be a chatbot, digital assistant, teaching tool or a translation system.
This research, and gestural human-machine interaction (G-HMI) more broadly, has significant potential for use in co-operative and social robotics, through explicit gestural interfaces, and robots with human-like implicit awareness of body language and gesture. This research’s focus on interaction via lexicalised and non-lexicalised signing as used by diverse individual visual-gestural communicators provides a basis for machine recognition and understanding that could underlie DST’s Kelpie project, as it may result in an approach to encoding multimodal language such that a robot could understand real-time messages communicated robustly in operational environments, with support for inter-signer variability in message production. By extension, it may also form a basis for more general encoding, able to capture signing, speaking, non-language gestures and abstract representations of the environment, for context-aware processing by autonomous systems.
In the news
Building better sign language technologies
Publications
Korte, Jessica, Bender, Axel, Gallasch, Guy, Wiles, Janet, and Back, Andrew (2020). A plan for developing an Auslan communication technologies pipeline. ECCV 2020: Computer Vision – ECCV 2020 Workshops, Glasgow, Scotland, 23–28 August, 2020. Cham, Switzerland: Springer International Publishing.https://doi.org/10.1007/978-3-030-66096-3_19
Get in touch
For more information about this project, please get in touch: