Guest Lecture: Quality-aware Ranking for Recommender Systems

Context:

Recommender systems commonly present users with a ranked list of items, known as a top-K answer. These rankings are typically generated by scoring candidate items based on information collected from users. Such scores pose a challenge for ranking items under inherent score uncertainty, which may arise from data unreliability and missing data. In this work, we address top-K queries over uncertain data in recommender systems. We explicitly model score uncertainty by representing scores as probability distributions rather than deterministic values. We first study static top-K recommendation under uncertain scores, where the recommendation size is fixed in advance and the answer is generated in a single step. We show that, under score uncertainty, the choice of ranking semantics should be aligned with the target quality measure. Based on this observation, we provide a formal analysis connecting quality measures and ranking approaches, and introduce rank-based methods for generating high-quality recommendations under uncertain scores. We then shift from the static top-K paradigm to an adaptive sequential recommendation setting under uncertain scores, where the answer is constructed incrementally rather than generated as a fixed list in a single step. In this setting, feedback observed after each recommendation can be used to update the recommendation process before selecting subsequent items. We propose adaptive ranking semantics, feedback-based update mechanisms, and stopping criteria that support the generation of high-quality recommendations while allowing the answer size to be determined dynamically. Finally, we study ordinal-aware learning of score distributions, aiming to improve the uncertainty estimates used by the recommendation process. Together, this work advances uncertainty-aware recommendation by showing how uncertain scores can be modeled and used to generate, adapt, and evaluate high-quality top-K recommendations.

This work is based on the following publications:

• Inbar Nachmani, Bar Genossar, Coral Scharf, Roee Shraga, Avigdor Gal: SLACE: A Monotone and Balance-Sensitive Loss Function for Ordinal Regression. AAAI 2025: 19598-19606

• Coral Scharf, Carmel Domshlak, Avigdor Gal, Haggai Roitman: A Rank-Based Approach to Recommender System's Top-K Queries with Uncertain Scores. Proc. ACM Manag. Data 3(1): 5:1-5:26 (2025)

• Dvir Cohen, Liad Domb, Avigdor Gal, Lior Ganon, Eliezer Gavriel, Omri Lazover, Coral Scharf, Bar Shterenberg: RecForUS: A Recommender System for Uncertain Scores. Proc. VLDB Endow. 18(12): 5267-5270 (2025)

• Coral Scharf, Roee Shraga, Avigdor Gal: Adaptive Sequential Recommendation under Uncertain Scores. Proc. VLDB Endow. 19 (2026), to appear

Presenter:

Professor Avigdor Gal, Professor of Data Science, Technion - Israel Institute of Technology

Presenter Bio:

Avigdor Gal is the Benjamin and Florence Free Chaired Professor of Data Science at the Faculty of Data & Decision Sciences, Technion - Israel Institute of Technology. Gal’s research focuses on data integration and process management and mining with about 200 publications, including multiple best paper and test-of-time awards. The research lab of Gal focuses these days on effective methods for embedding sensor and human input in generating machine learning models that can improve upon the separate use of algorithms and humans. Over the past decade artificial intelligence (AI) has become a key ingredient in our daily routine. Learning from sensor data using expert human knowledge is an exciting field of research, which requires careful analysis of data and knowledge noise and the costly acquisition of high-quality data and knowledge. We focus on the use of knowledge graph to encode expert knowledge and sensor data to fine-tune it. We have used expert models successfully in applications to electricity consumption, urban environment 3D models, hospital schedules, smart transportation, and the dairy industry.