Subscribe to Email Updates

Collaborative Filtering is the new Customer Segmentation

Posted by Bala Deshpande on Tue, Apr 19, 2016 @ 10:38 AM

There was recently a very interesting article about how Netflix thinks demographic data is irrelevant to marketing. In social sciences, it is understood that sometimes, the variation between members of the same demographic can be larger than the average variation between two demographies. As a simple example to drive it home, think about soccer fans for the English Premier League. Manchester United has fans across the world, among many ethnicities - where as within England a ManU fan would be dissimilar (and would hate to be compared) to say, Liverpool, although demographically they may be identical!

Netflix found a similar trend in the fandom of Japanese Anime shows. The fall out from this type of understanding of behavior makes age, gender, race, nationality and other "traditional" demographics less important in todays data flooded world.

We all know how the recommender engine was popularized by Netflix. For the type of product they sell, apparently this makes the most sense. People may be separated by continents, but they may still like the same genre of movies. In this context a collaboritive filter acts like a one dimensional clustering model. The dimension being similar tastes. As seen in the figure, the intra-example distance (d1) between two points in the same cluster, can be much larger than the inter-example distance (d2) between two points of different clusters. This should not be surprising - because on a dimension (or attribute) that contributes to the similarities, the red and green examples (in d2) are more similar. When we run a recommender engine this attribute is the only one that is considered. collaborative-filtering-as-clustering.png

Does this mean that traditional demographics based analysis is no longer relevant? It depends. If we have enough granular data about our customers - including their day to day likes and dislikes, attitudes etc, then demographics indeed will play a very minor role (if at all) in helping us segment our customers. However, most businesses today do not have the luxury of such data. Further more, i think that as the purchase value of the product is larger, then demographics will certainly be important - think about car or home purchases. These are not dictated by day to day changes in attitudes or behavior. Here demographics may still matter.

However in the world on Likes and tweets, instant behavioral shifts do matter and if your business depends on such shifts, you are better off using collaborative filtering type approaches to segmenting your customers. 

What would be more valuable is if you can combine data from different sources - surveys, social media updates and other disparate places and then build your segmentation models. Those would be much richer than plain vanilla demog models.

Download Market Segmentation Case Study


Topics: clustering technique, collaborative filtering