Scalable and Lightweight Content-based Recommender Systems

The School of EECS is hosting the following PhD Progress Review 1 Confirmation Seminar:

Scalable and Lightweight Content-based Recommender Systems

Speaker: Hung Vinh Tran
Host: A/Prof. Sen Wang

Abstract: Content-based recommender systems (CRSs) play a crucial role in assisting users to navigate the vast information available on the Internet. By generating personalized recommendations, they are indispensable in domains like e-commerce, entertainment, and online content, driving user engagement and business revenue. However, as the scale of recommendation tasks grows, so does the demand for efficient model design. First, the large embedding tables for sparse categorical feature representations quickly create a major memory bottleneck,  complicating deployment on devices with different memory budgets, as most lightweight models require fine-tuning for each budget constraint. Second, the resource requirements for training on massive datasets introduce significant overhead, especially when frequent updates are necessary to reflect shifting user preferences. Third, integrating multimodal data, such as text, images, and audio, holds great promise but requires novel approaches to manage the complexity and computing cost associated with these diverse modalities.

In this report, we first present the proposed solution for the first challenge -- Shapley Value-guided Embedding Reduction (Shaver). With Shaver, we view the model pruning problem from a cooperative game perspective and quantify each embedding parameter's contribution with Shapley values to facilitate the pruning process. To address the inherently high computation costs of Shapley values, we provide an efficient and unbiased method to estimate Shapley values of a CRS's embedding parameters. Moreover, in the pruning stage, we employ a field-aware codebook to alleviate the information loss in the traditional zero-out approach. Subsequently, we provide the technical pathways to solve the remaining two challenges through the candidacy journey and introduce how they will jointly contribute to a more efficient and sustainable pipeline to today's recommender systems.

Bio: Hung Vinh Tran (Trần Vinh Hưng) is a first-year PhD student in Computer Science at the University of Queensland, Australia. He is currently working on model efficiency for recommender systems, under the supervision of Dr. Rocky Tong Chen and Prof. Hongzhi Yin. Previously, he was a researcher and engineer at Cinnamon AI. He obtained his Bachelor's Degree from the Honors Program of the Department of Information Technology, Ho Chi Minh City University of Science, Vietnam National University (VNU-HCMUS).