Decentralized Learning for On-device Recommendation

The School of EECS is hosting the following PhD Milestone 3 Thesis Review Seminar:

Decentralized Learning for On-device Recommendation

Speaker: Ruiqi Zheng
Chair: A/Prof Sen Wang

Abstract: Recommender systems have become an integral part of our digital lives, providing personalized suggestions across various domains such as e-commerce and video streaming by filtering massive datasets to match user preferences. To make room for privacy and efficiency, the deployment of many recommender systems is experiencing a shift from central servers to personal devices, where the federated recommender systems (FedRecs) and decentralized collaborative recommender systems (DecRecs) are arguably the two most representative paradigms. While both leverage knowledge (e.g., gradients) sharing to facilitate learning local models, FedRecs rely on a central server to coordinate the optimization process, yet in DecRecs, the knowledge sharing directly happens between clients. Despite the advancements of DecRecs, they face unique challenges, including constrained local storage, inflexible architectures, vulnerability to poisoning attacks, and performance degradation.

This thesis introduces several innovations to address these challenges. Firstly, it proposes a personalized elastic embedding learning framework for DecRecs to counteract model invisibility due to the constrained local storage on the device side. This method generates a personalized recommendation model for devices with various memory budgets in a once-for-all manner, adapting to new or dynamic budgets, and addressing user preference diversity by assigning personalized embeddings for different groups of users. Secondly, this thesis proposes a decentralized collaborative learning with adaptive reference data framework to address the inflexible architecture issues caused by the homogeneous model requirements in the original communication manner. During the communication between two users, they exchange soft decisions based on a combined set of their adaptive reference data instead of raw gradients.
Thirdly, this thesis proposes model poisoning with adaptive malicious neighbors and its countermeasures to investigate the vulnerability of DecRecs. The attack method effectively boosts target items’ ranks with several adversaries that emulate benign users and transfers adaptively crafted gradients conditioned on each adversary’s neighbors, while the countermeasure based on user-level gradient clipping with sparsified updating effectively protects DecRecs. Finally, knowledge graphs are utilized as auxiliary information to enhance model performance by providing semantic relationships to counteract the performance degradation issue.

Bio: Mr. Ruiqi Zheng received a B.E. in Computer Science and Technology in 2021 from Southern University of Science and Technology. He started his Ph.D. in computer science under the supervision of Prof. Hongzhi Yin and Dr. Rocky Chen. His main research interests are decentralized learning and recommender systems.