When meeting someone new, first impressions are crucial: the expression “You never get a second chance to make a first impression” indicates how people often make snap judgments. The same holds true when installing and trying a new application for the first time. Just as you’d negatively judge someone showing up late to a first […]
When meeting someone new, first impressions are crucial: the expression “You never get a second chance to make a first impression” indicates how people often make snap judgments. The same holds true when installing and trying a new application for the first time. Just as you’d negatively judge someone showing up late to a first date, what would you think of an application that gives you far off the mark recommendations, even though you explicitly stated your preferences?
In this context, online music streaming services must quickly come up with accurate content that new subscribers are likely to enjoy. However, while we know recommendation algorithms work well for users who have had enough interactions with the service (and that their preferences are well known), recommending music to new users remains a largely unresolved challenge for the research community. In scientific literature, this is commonly referred to as the user cold start problem.
In order to match every new subscriber with Deezer, the Recommendation and Research teams have worked together to improve music suggestions specifically for these new users. In this post, we describe the system we recently deployed at Deezer to address this issue.
Broadly speaking, recommendation systems can use two types of strategies (or combinations thereof) to find relevant content for users:
In recent industrial-level work, latent models for collaborative filtering proved to be efficient. In a nutshell, they consist of learning a vector space representation of users and items (i.e. a compressed way to represent users), which we call embeddings. In this context, proximity should reflect user preferences, typically via the factorization of a user-item interaction matrix. Such user preference representations are very convenient in a production environment. At Deezer, we use them to populate the playlist recommendation carousel for instance, which is a swipeable list of playlists on the homepage, as illustrated below.
Yet, in the real world of music recommendations, we are facing three issues with these latent models:
In light of this practical industrial point of view, we need to tackle this cold start issue. More formally: how can we incorporate new users into an existing latent space model based on already registered users data?
To predict new user embeddings, reflecting what kind of music these users will like, we propose to train a machine learning model specifically dedicated to this task. We gather users that are now senior and active enough in our database; in contrast to the cold users, referred to as warm users here. We retrieve the information as if it was their registration day, ignoring all their subsequent interactions. That is to say, we simulate the cold start situation for warm users — who are actually well represented with an embedding. We use diverse information such as country and age, as well as interactions (if any), both positive or negative, like streams, additions to favorites, bans or dislikes, searches, etc. We enrich these features using the warm embeddings of users and items — they can be tracks, artists, albums or playlists. For each type of interaction and music entity, embedding vectors are averaged, as illustrated in the following figure. We obtain fixed-size representations for both demographics and interactions, meaning they are independent of the number of modalities or interactions, which is crucial for scalability in a production environment.
As described in the previous figure, we then propose a three-hidden layers neural network aiming at predicting user embeddings. We train the model on warm users by iteratively minimizing the mean squared error between the predicted user embeddings and their actual value by stochastic gradient descent. Once we obtain a fully trained model, one could directly provide fully personalized music recommendations from this predicted user embedding by retrieving the most similar tracks for each user via an exact or approximate nearest neighbor search.
As we empirically show in our experiments (see the paper for full detail), leveraging similarity directly on the predicted user embedding can still lead to noisy results for users with very little to no usage data. As a consequence, in production, our system adopts a semi-personalized recommendation strategy instead, as described in the following figure:
This approach is all the more relevant in our industrial stack at Deezer as we can adapt the type of recommended content for new users here. Indeed, we can better recommend music items such as tracks, artists, albums, but also some of the personalized cards present in the carousel of our homepage such as the new release or the discovery card.
For offline experiments (experiments prior to A/B testing), we extracted a dataset of fully anonymized Deezer users, composed of 70,000 warm users for the training dataset, and 30,000 cold users for the testing and validation ones. We gathered socio-demographic and interaction information available by the end of registration day. We compared the performances of our approach with two selected models from the state of the art, DropoutNet and MeLU, by measuring their ability to predict the future streams of new users, i.e. after registration day. Our semi-personalized approach performed significantly better on the standard ranking metrics for recommendation we considered: standard recall (i.e. the fraction of relevant tracks that were retrieved), precision (i.e. the fraction of relevant tracks among the retrieved ones), ndcg (i.e. the Normalized Discounted Cumulative Gain, which is a measure of ranking quality that takes into account the order of the retrieved tracks).
We launched a large-scale A/B test to assess the ability to recommend personalized playlists in the carousel of the Deezer homepage that we mentioned at the beginning of this post. In the following figure, we can observe that our new semi-personalized system led to significant improvements of the relative display-to-stream and display-to-favorite rates. The display-to-stream rate refers to the proportion of users who streamed when the playlists were displayed on their homepage: in the graph below, we see that our strategy outperformed the previous approach in production by more than 10 percentage points. The display-to-favorite rate refers to the proportion of users who added the playlists displayed to their favorites: here our strategy outperformed the previous approach by more than 20 percentage points. Such results validate the relevance of our proposed system, and emphasize its practical impact on Deezer 🎉.
Our offline and online evaluations show that we managed to improve recommendations for new users whose embeddings were lacking in our production stack. Nevertheless, here we solely focused on improving our very first recommendations with all the information available on Deezer for brand new users, and not the reaction (positive or negative) to these recommendations, which begs the question, why not take this better into account to improve user representations? In the future, we would like to include temporal aspects in our approach, and fine-tune our recommendations. We have also thought about more sophisticated approaches at different stages of our system, such as proposing graph methods for user clustering, which show promising results in the scientific literature. To be continued!
For further information, please refer to our paper:
A Semi-Personalized System for User Cold Start Recommendation on Music Streaming Apps Léa Briand, Guillaume Salha-Galvan, Walid Bendada, Mathieu Morlon and Viet-Anh Tran. Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 2021), 2601-2609.
If you would like to help us improve the Deezer experience for millions of users, check our open positions and join one of our teams!