Multi-modal Data Modelling with Awareness of Efficiency, Reliability, and Privacy

The School of ITEE is hosting the following thesis review seminar:

Multi-modal Data Modelling with Awareness of Efficiency, Reliability, and Privacy

Speaker: Mr Pengfei Zhang
Host: Prof Helen Huang

Abstract: Due to the inherent nature of data acquisition and generation, certain concerns surrounding responsible multi-modal data modelling are raised, including efficiency, reliability, and privacy. First of all, existing methods usually pursue better performance with deeper and larger architectures and high-dimensional features, putting significant pressure on computation and memory. Second, they also highly rely on large-scale datasets with high-quality annotations. However, in real-world, it is a tough job to label a large-scale dataset. How to conduct multi-modal data modelling without supervised information remains a challenge. Lastly, user-generated data might contain sensitive information, putting data owners under potential risks of privacy leakage by unauthorized data exploration. In this thesis, we introduce several novel approaches, attempting to fulfil responsible multi-modal data modelling.  Our contribution includes: (1) To facilitate efficient and effective data query, we propose an end-to-end GCN-based deep hashing learning framework to learn compact binary representations by exploiting reliable data relationships and effectively associating data of different modalities. (2) Furthermore, we consider learning a light-weight model by taking advantages of multi-level knowledge from cross-task teachers with online knowledge refinery.  (3) Beyond the scope of efficiency and effectiveness, we further propose to proactively protect data contents before data release by injecting imperceptible perturbations, preventing malicious data exploration. (4) Finally, considering that data might have been used, we design a novel machine unlearning method to enable data removal from a trained model responding to deletion requests, without retraining the model from scratch or full access to the original training dataset.  Our research fulfils the diverse requirements for trustworthy data modelling.

Bio: Pengfei Zhang is a Ph.D. candidate from the School of ITEE at the University of Queensland under the supervision of Prof Helen Huang. He received his degree of Master in Computer Science from the Shandong University, China. His research interests include privacy protection, robust learning, multi-model learning and information retrieval.