The School of EECS is hosting the following PhD Progress Review 3 Seminar:
Evidence-based Multimodal Dialogue with Reasoning
Speaker: Fenghua Cheng
Host: Dr Xin Yu
Abstract:
With the rapid development of multimodal intelligent agents, models are increasingly capable of processing diverse information from text, images, and videos. However, processing multimodal inputs does not necessarily guarantee that model responses are reliably grounded in task-relevant evidence. To support reliable multimodal reasoning, an intelligent agent should be able to: 1) understand evidence embedded in complex events and interactions; 2) integrate evidence distributed across different levels of granularity together with external knowledge; 3) identify and correct conflicting evidence across modalities; and 4) actively acquire latent evidence that is not directly observable. Nonetheless, existing multimodal dialogue research often assumes that evidence is relatively straightforward, consistent, explicit, and directly accessible, leaving these more realistic evidence-grounding challenges underexplored. To address these issues, this PhD research investigates four key aspects: 1) grounding dialogue generation in complex event evidence; 2) integrating hierarchical visual evidence and external knowledge for multimodal reasoning; 3) verifying conflicting visual-textual evidence in multi-turn dialogue; and 4) identifying latent knowledge blind spots through active interaction. The overall aim of this research is to advance multimodal intelligent agents towards more reliable evidence-grounded reasoning and interaction.
Biography:
Fenghua Cheng is a Ph.D. student at the school of EECS in the University of Queensland, under the supervision of Prof. Xue Li and Prof. Helen Huang. He obtained his Bachelor of Computer Science from both Shandong University, China and Australian National University. He completed his Master of Computing at Australian National University. His research interests include multi-modal machine learning and vision-language learning.
About Data Science Seminar
This seminar series is hosted by EECS Data Science.
Venue
Room 78-411 or https://uqz.zoom.us/j/4446681535