collaborative filtering

There was recently a very interesting article about how Netflix thinks demographic data is irrelevant to marketing, because its collaborative filtering based recommender engines can do without it. In social sciences, it is understood that sometimes, the variation between members of the same demographic can be larger than the average variation between two demographies. As a simple example to drive it home, think about soccer fans for the English Premier League. Manchester United has fans across the world, among many ethnicities – where as within England a ManU fan would be dissimilar (and would hate to be compared) to say, Liverpool, although demographically they may be identical!

Netflix found a similar trend in the fandom of Japanese Anime shows. The fall out from this type of understanding of behavior makes age, gender, race, nationality and other “traditional” demographics less important in todays data flooded world.

We all know how the recommender engine was popularized by Netflix. For the type of product they sell, apparently this makes the most sense. People may be separated by continents, but they may still like the same genre of movies. In this context a collaborative filter acts like a one dimensional clustering model. The dimension being similar tastes. The intra-example distance (d1) between two points in the same cluster, can be much larger than the inter-example distance (d2) between two points of different clusters. This should not be surprising – because on a dimension (or attribute) that contributes to the similarities, the examples (separated by d2) could be more similar. When we run a recommender engine this attribute is the only one that is considered. 

Does this mean that traditional demographics based analysis is no longer relevant for things like segmentation? It depends. If we have enough granular data about our customers – including their day to day likes and dislikes, attitudes etc, then demographics indeed will play a very minor role (if at all) in helping us segment our customers. However, most businesses today do not have the luxury of such data. Further more, i think that as the purchase value of the product is larger, then demographics will certainly be important – think about car or home purchases. These are not dictated by day to day changes in attitudes or behavior. Here demographics may still matter.

However in the world on Likes and tweets, instant behavioral shifts do matter and if your business depends on such shifts, you are better off using collaborative filtering type approaches to segmenting your customers. 

What would be more valuable is if you can combine data from different sources – surveys, social media updates and other disparate places and then build your segmentation models. Those would be much richer than plain vanilla demographic models.

Originally posted on Tue, Apr 19, 2016 @ 10:38 AM

Photo by Dollar Gill on Unsplash

No responses yet

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.