AutoML for On-device Recommender Systems

The School of ITEE is hosting the following PhD confirmation seminar:

AutoML for On-device Recommender Systems

Speaker: Ruiqi Zheng
Host: A/Prof. Hongzhi Yin

Abstract: To capture users' changing interests in real-time and reduce network latency, compressing cumbersome recommender systems (RecSys) trained on the cloud and deploying them to resource-limited devices for real-time inference has attracted considerable interest. Existing solutions for on-device RecSys generally assume that devices with the same memory budget share the same compressed model, and a device's budget is fixed. This assumption ignores diversified user preferences among devices with the same memory budget and requires a time-consuming retraining process for a new memory budget. Additionally, device budgets are both heterogeneous and dynamic, as different devices have various budgets, and available resources on one device change in real-world scenarios. It is impractical to manually design a model for every device with a heterogeneous and dynamic budget. Automated Machine Learning (AutoML) for on-device RecSys aims to automatically design and update on-device RecSys to tackle the aforementioned gap. In this manner, we propose a Personalized Elastic Embedding Learning framework (PEEL) for on-device recommendation, which automatically generates personalized embeddings for devices with various memory budgets and can efficiently adapt to new or dynamic budgets. Specifically, it pretrains over the global user-item interaction instances to generate the global embedding table and clusters users into groups. Then, it refines the embedding tables with local interaction instances within each group. Personalized elastic embedding is generated from the group-wise embedding blocks and their weights, which indicate the contribution of each embedding block to the local recommendation performance. Given a memory budget, PEEL efficiently generates personalized elastic embeddings by selecting embedding blocks with the largest weights, making this solution efficiently adapt to the setting of a dynamic memory budget on a device. A diversity-driven regularizer is implemented to encourage the expressiveness of embedding blocks, and a controller is utilized to optimize the weights. Extensive experiments were conducted on two public datasets, which showed that PEEL yields superior performance on devices with heterogeneous and dynamic memory budgets.

Speaker Bio: Ruiqi Zheng is a Ph.D. student at the School of ITEE at the University of Queensland, supervised by A/Prof. Hongzhi Yin and Dr. Tong Chen. He received his bachelor's degree in Computer Science and Technology from the Southern University of Science and Technology in China. His research interests include recommender systems and AutoML.