Reducing Offline Evaluation Bias in Recommendation Systems

Arnaud de Myttenaere (1) , Bénédicte Le Grand (2) , Boris Golden (3) , Fabrice Rossi (1)

1 SAMM - Statistique, Analyse et Modélisation Multidisciplinaire (SAmos-Marin Mersenne)
2 CRI - Centre de Recherche en Informatique de Paris 1
3 Viadeo - Viadeo SA

Recommendation systems have been integrated into the majority of large online systems. They tailor those systems to individual users by filtering and ranking information according to user profiles. This adaptation process influences the way users interact with the system and, as a consequence, increases the difficulty of evaluating a recommendation algorithm with historical data (via offline evaluation). This paper analyses this evaluation bias and proposes a simple item weighting solution that reduces its impact. The efficiency of the proposed solution is evaluated on real world data extracted from Viadeo professional social network.

Mots clés: recommendation systems; offline evaluation; evaluation bias; covariate shift

Domaines: Machine Learning [stat.ML]

Arnaud de Myttenaere, Bénédicte Le Grand, Boris Golden, Fabrice Rossi. Reducing Offline Evaluation Bias in Recommendation Systems. 23rd annual Belgian-Dutch Conference on Machine Learning (Benelearn 2014), Jun 2014, Bruxelles, Belgium. pp.55-62. ⟨hal-01017734⟩