Informative Representation Learning with Multimodal Data Fusion

The School of EECS is hosting the following PhD Progress Review 1 Confirmation Seminar:

Informative Representation Learning with Multimodal Data Fusion

Speaker: Jinxiang Wang
Host: Prof. Xue Li

Abstract: In the domain of multimodal data analysis, multimodal deep learning aims to develop advanced intelligence for understanding multiple modalities such as visual, text, audio, sensors and table data. The representation and integration of heterogeneous data from various sources present a significant challenge. This research addresses the imperative need for informative representation learning through multimodal data fusion techniques. Our study preserves the task-related information inherent in different data modalities and aims to enhance the understanding of complex multimodal interactions. we have established a comprehensive framework for learning informative representations from multimodal data, with a focus on capturing both global patterns and fine-grained details present across multiple data sources. In our clinical settings, the proposed methods facilitate the extraction of interpretable and actionable medical knowledge from Unified Medical Language System (UMLS) and encode the fine-grained representations from both medical chest x-ray images and associated reports. By extensive experiments on three large public available datasets, our approach is demonstrated to improve the thorax disease classification results. Ultimately, the outcomes of this study hold the potential to significantly impact various healthcare applications, including disease diagnosis, treatment planning, and patient management.

Bio: Jinxiang Wang is a Ph.D. student at the School of EECS at the University of Queensland, under the supervision of Prof. Xue Li and Associate Prof. Janet Hou. He completed his Bachelor in computer science at Wuhan University, China and his Master in computer science at the University of Melbourne. His research interests include representation learning, medical data analysis and multimodal machine learning with applications in natural language processing and healthcare.