Improving the Efficiency and Effectiveness of Dense Retrieval using Adaptive Embedding Dimension Prediction

The School of EECS is hosting the following MPhil Progress Review 1 Seminar:

Improving the Efficiency and Effectiveness of Dense Retrieval using Adaptive Embedding Dimension Prediction

Speaker: Tharita Tipdecho
Principal advisor: Professor J. Shane Culpepper
Co-advisor: Dr. Joel Mackenzie

Abstract:  Matryoshka Representation Learning (MRL) is a new technique that nests embeddings of varying length by biasing weights towards the front of an embedding. MRL offers the benefit of producing multiple embeddings of varying length for the same training cost as traditional dense retrieval techniques, providing flexibility in balancing computational efficiency and performance. Current dense retrieval systems typically utilize fixed-size embedding dimensions, which may ignore important interactions between documents and queries. In this work, we show that optimal ranking performance can be achieved without requiring the full-length embeddings, which leads to improved computational efficiency. We explore how to combine MRL with a dense retriever to enhance the performance of passage ranking tasks. By leveraging MRL, our approach enables embeddings of varying size to be extracted in a single model architecture. To test our hypothesis, we train a classifier designed to predict the most suitable embedding length for each query at retrieval time. Our model is trained with the MS MARCO dataset, a large-scale benchmark for passage retrieval. We conduct extensive experiments to assess the viability of embedding dimension prediction in real-world search scenarios and explore its impact on retrieval efficiency and effectiveness.