Improving Discriminative Retrieval Models Using Generative Tasks

The Data Science discipline in EECS is hosting the following guest seminar

Improving Discriminative Retrieval Models Using Generative Tasks

Speaker: Professor Shane Culpepper, RMIT University

Abstract: Search and recommendation systems have a long history of applying either discriminative or generative modelling to retrieval and ranking tasks. Recent developments in transformer architectures and multi-task learning techniques have dramatically improved our ability to train effective neural models capable of resolving a wide variety of tasks using either of these paradigms. In this talk, we will discuss a novel multi-task learning approach which can be used to produce more effective neural ranking models. The key idea is to improve the quality of the underlying transformer model by cross-training a retrieval task and one or more complementary language generation tasks. By targeting the training on the encoding layer in the transformer architecture, our experimental results show that the proposed multi-task learning approach consistently improves retrieval effectiveness on the targeted collection and can easily be retargeted to new ranking tasks. We provide an in-depth analysis showing how multi-task learning modifies model behaviours, resulting in more general models.

Bio: Professor Shane Culpepper is the Director for the Centre for Information Discovery and Data Analytics (CIDDA) at RMIT University in Melbourne, Australia. His research focuses primarily on Search and Recommendation Systems. Over his 14 year career at RMIT, Professor Culpepper has supervised 19 PhD students, 15 of which have now graduated, and co-authored a total of 113 papers with 127 different research collaborators on problems such as algorithm efficiency and scalability, new machine learning algorithms for search and recommendation systems, and evaluating search and recommendation engine quality. Professor Culpepper is also an active member in the international research community. In the last 5 years, he has been a program co-chair for international conferences such as SIGIR and CIKM, and co-organized conferences such as WSDM and SWIRL.

The seminar will be 40 minutes followed by a 15-minute Q&A

DS group Morning Tea will be served on the level 4 balcony in GP South following the seminar