Towards Robust End-to-end Autonomous Driving Neural Networks

The school of EECS is hosting the following Progress Review 1 seminar:

Towards Robust End-to-end Autonomous Driving Neural Networks

Speaker: Huitong Yang

Abstract.

Autonomous driving systems must operate reliably under diverse and unpredictable real-world conditions, including adverse weather, illumination changes, sensor noise, and cross-domain deployment shifts. These factors introduce significant distribution gaps between training and deployment, causing error accumulation and a lack of joint optimization across modules.  Test-time adaptation (TTA) offers a promising direction by enabling models to adapt online during inference. However, existing test-time adaptation (TTA) methods often fail in high-variance tasks like 3D object detection due to unstable optimization and sharp minima. While recent model merging strategies based on linear mode connectivity (LMC) offer improved stability by interpolating between fine-tuned checkpoints, they are computationally expensive, requiring repeated checkpoint access and multiple forward passes.

To address these gaps, we introduce CodeMerge, a lightweight and scalable model merging framework that bypasses these limitations by operating in a compact latent space. Instead of loading full models, CodeMerge represents each checkpoint with a low-dimensional fingerprint derived from the source model’s penultimate features and constructs a key-value codebook. We compute merging coefficients using ridge leverage scores on these fingerprints, enabling efficient model composition without compromising adaptation quality. These efforts collectively advance the generalization capability of autonomous driving systems under domain shifts.

Bio: Mr Huitong Yang is a PhD candidate from the Data Science group at the School of Electrical Engineering and Computer Science, the University of Queensland (UQ), Australia. His research focuses on Autonomous Driving and Embodied AI, under the supervision of Professor Zi Huang and Dr. Yadan Luo.