Neural Re-ranking in Multi-Stage Recommender Systems: A Review

As the final stage of the multi-stage recommender system (MRS), re-ranking directly affects users’ experience and satisfaction by rearranging the input ranking lists, and thereby plays a critical role in MRS. With the advances in deep learning, neural re-ranking has become a trending topic and has been widely adopted in industrial applications. This review aims at integrating re-ranking algorithms into a broader picture, and paving the way for more comprehensive solutions for future research. For this purpose, we first present a taxonomy of current methods on neural re-ranking. Then, we describe these methods along with their historical development according to their objectives. The network structure, personalization, and complexity are also discussed and compared. Next, we provide a benchmark for major neural re-ranking models and quantitatively analyze their re-ranking performance. Finally, the review concludes with a discussion on future prospects of this field. A list of papers discussed in this review, the benchmark datasets, our re-ranking library LibRerank, and detailed parameter settings are publicly available at https://github.com/LibRerank-Community/LibRerank.