Toward Memory-Efficient Recommender Systems

The School of EECS is hosting the following PhD Progress Review 1 Confirmation Seminar:

Toward Memory-Efficient Recommender Systems

Speaker: Jason Xurong Liang
Host: A/Prof. Sen Wang

Abstract: Recommender systems have exerted a great impact on generating accurate personal recommendations in e-commerce applications, thus effectively boosting the financial revenue of online businesses. However, as the scale of recommendation services in e-commerce applications enlarges quickly, concerns about the memory efficiency of recommender systems are raised. Modern recommender systems often represent each entity (commonly a user/item) using a unique vector embedding with a fixed dimension size (e.g., 128). Due to the enormous number of users and items to be modeled in the recommender systems, the embedding table, which stores all entity embeddings, becomes the heaviest component of the recommender system. Thus, creating difficulty in devising state-of-the-art recommender systems on memory-restrained devices. After analyzing on-the-shelf lightweight embedding optimization frameworks, three challenges obstructing the pathway to memory-efficient recommender systems have been identified: 1) Maintaining the recommendation accuracy of the lightweight recommender systems; 2) Enabling wide adaptability of lightweight recommender systems to various memory budgets without training from scratch; 3) Optimizing the runtime memory footprint of lightweight recommender systems.

At present, CERP and LEGCF have been proposed to address the first challenge. CERP deploys two balanced meta-codebooks and performs joint pruning to reduce the total parameter size of the embedding layer. A pruning regularizer is introduced to encourage the two sparsified meta-codebooks to encode information that is mutually complementary. LEGCF is a graph-based lightweight recommender framework that devises a single meta-codebook along with a learnable assignment matrix. To jointly optimize these two heavily entangled components, aside from learning the meta-embeddings by minimizing the recommendation loss, LEGCF further performs efficient assignment update by enforcing a novel semantic similarity constraint and finding its closed-form solution based on matrix pseudo-inverse. Extensive experiments on public benchmark datasets have verified the effectiveness of the two proposed work in retaining excellent recommendation performance.

Bio: Jason Xurong Liang is a first-year Ph.D. student at the University of Queensland, Australia. Prior to his candidacy, he earned a Bachelor of Computer Science degree with first-class honors from the University of Queensland. Jason is currently working on the research topic of lightweight recommender system, advised by Dr. Rocky Tong Chen and Prof. Hongzhi Yin. Looking at his research interest on a broader scale, he is keen to explore ideas and fun stuff in data mining, recommender systems, and user modeling. The three main things he often does when he is not hitting the books or sitting in front of a computer are traveling, fishing, and watching movies.