Sequential recommendation: A study on transformers, nearest neighbors and sampled metrics

Sequential recommendation problems have received increased research interest in recent years. In such scenarios, the task is to suggest items to users to consume next, given their past interaction history, e.g., the next movie to watch or the next item to place in the shopping cart. A number of machine learning models were proposed recently for the task of sequential recommendation, with the latest ones based on deep learning techniques, in particular on Transformers. Given the often surprisingly competitive performance of simpler nearest-neighbor methods for the related problem of session-based recommendation, we investigate the use of nearest-neighbor methods for sequential recommendation problems. Our analysis on four datasets shows that nearest-neighbor methods achieve comparable or better performance than the recent Transformer-based bert4rec method on two of them. However, the deep learning method outperforms the simple methods for the two larger datasets, confirming previous hypotheses that neural methods work best when more data is available. As a further result of our experiments, we found additional evidence that sampled metrics must be used with care, as they may not be predictive of an algorithm ranking that would be observed with the non-sampled, full evaluation.