Self-Supervised Recommender Systems

The School of ITEE is hosting the following thesis review seminar:

Self-Supervised Recommender Systems

Speaker: Mr Junliang Yu
Host: A/Prof Hongzhi Yin

Abstract: Nowadays, recommender systems have become an indispensable component of E-commerce and content-sharing platforms, for creating delightful user experiences whilst driving incremental revenue. As most modern recommender systems are powered by deep neural architectures, to take advantage of them, the required volume of training data is far larger than ever needed. However, data acquisition in recommender systems is costly because most users can only consume a tiny fraction of provided items. In consequence, the resulting data sparsity issue often bottlenecks deep recommendation models to reach their full potential.

Self-supervised learning (SSL), as a nascent technique for reducing the dependency on manual labels, recently has achieved immense success in many fields. Its fundamental idea is to extract intrinsic supervisory signals from unlabelled data through well-designed self-supervised tasks and data augmentations. Since the principle of SSL is in accord with recommender systems' needs for more annotated data, we propose self-supervised recommender systems that can break the data sparsity bottleneck and make full use of neural architectures.

In this thesis, we systematically investigate self-supervised recommender systems. As the first to define and formulate this concept, we not only offer a panorama of self-supervised recommender systems but identify several critical technical challenges and provide the corresponding solutions. Specifically, our research consists of the following three parts. Firstly, we argue that recommender systems are the convergence of various data which often appear as different graphs (e.g., social network) and are of locally vague semantics. The ready-made SSL schemes in other fields for dealing with homogeneous and human-readable data cannot be seamlessly transplanted to recommendation. We thus explore multi-view SSL schemes for recommendation and propose MHCN which can perform view-wise self-supervision while capturing the implicit semantics through hypergraph modelling. Secondly, we notice that previous SSL schemes in other fields regard every instance as a single class and only seek self-supervision signals from its own augmentations. However, in recommender systems, many users have similar preferences. It is reasonable to utilize the self-supervision signals from other users. To this end, we combine SSL with semi-supervised learning and develop SEPT which relies on a semi-supervised learning framework to discover similar users and then exploit them through the proposed multi-instance SSL scheme. Thirdly, given the effectiveness of SSL for improving recommendation, it is significant to know what on earth underlies the performance gains. We first empirically prove that data augmentations do not play a crucial role in self-supervised recommenders as thought and reveal what really matters is the contrastive self-supervised optimization objective. On top these findings, we then propose SimGCL and XSimGCL, which are simple yet powerful self-supervised recommendation models, outperforming state-of-the-art recommendation models by a large margin in terms of both recommendation accuracy and training efficiency. 

Bio: Junliang Yu is a Ph.D. candidate from the School of ITEE at the University of Queensland under the supervision of A/Prof Hongzhi Yin. He received his degrees of Master and Bachelor in Software Engineering from Chongqing University, China. His research interests include recommender systems,  graph learning, self-supervised learning and tiny machine learning.