Empowering Human-Centric Understanding with MLLMs: From Single View to Multi-View

 

Empowering Human-Centric Understanding with MLLMs: From Single View to Multi-View

 

Speaker: Tianchen Guo

 

Abstract: Human-centric understanding is increasingly utilizing Multimodal Large Language Models (MLLMs), yet current systems are fundamentally limited by their reliance on single-view paradigms that struggle with occlusions in dynamic, real-world environments. Existing multi-image evaluation frameworks employ a static "bag of frames" approach that rewards single-view pattern recognition, failing to test genuine 3D cross-view fusion and exposing a systemic "single-view bias" in current architectures. To address these scientific gaps, this research introduces PLNet-12 to quantify fine-grained temporal understanding, alongside diagnostic benchmarks to rigorously diagnose cross-view reasoning capabilities, single-view insufficiency, and redundancy-induced distraction degradation. Future objectives focus on achieving genuine multi-view fusion through novel cross-view attention mechanisms and biomechanics-guided tuning, ultimately facilitating automated AI coaching and precise disability classification for real-world applications such as the Brisbane 2032 Olympic Games and assistive healthcare technologies.

 

Bio: Tianchen Guo is a second-year Ph.D. student at the School of EECS, University of Queensland, under the supervision of AsPr Xin Yu and Dr Miao Xu. His research centres on human-centric understanding.