data science challenge

The gamification of data science – creating a data science challenge – into a contest for prize money tends to confuse businesses about what it takes to become truly proficient in using data and analytics. These competitions often downplay the complicated system of activities that are prerequisities for success with data.

It is well recognized today that data science is a combination of three totally separate areas of expertise: math and statisticscomputing and/or hacking and finally domain knowledge. Finding one person who combines all three is also correctly recognized as finding the unicorn – the data science challenge! Data science competitions however focus mainly on one skill: computing/coding. While the winner of a competition may understand the math behind the algorithms they are coding, they are not necessarily expert statisticians or mathematicians. And they are very unlikely to be business domain experts.

Some companies today believe that a good way to outsource data science is through these competitions. While they may find success in solving a specific business problem through this mechanism, it does not help them to institutionalize data and analytics into their organization. 

According to Davenport“Companies … that compete analytically don’t entrust analytical activities to just one group within the company or a collection of disparate employees across the organization”. Unfortunately this is what gamification of data science tends to encourage. Successful deployment of data science for business requires careful planning and strategic thinking. Launching a competition or a data science challenge requires a business to address the following three pre-requisites, assuming that a specific high value business problem has already been identified:

  1. Deciding on what data should be made available
  2. Deciding what is the target or response variable
  3. Deciding what constitutes success

Each of these questions can only be answered by someone who has deep knowledge of the business or the domain and an advanced level of statistical expertise. Once a business has successfully addressed the above three questions, it then makes sense to open up the problem as a analytical challenge that can be solved with hacking skills.

But many businesses today are in the early stages of their maturity with data and analytics. While they will have people who possess domain knowledge, it is unlikely that these same people will also be statisticians. That is when it would make sense for companies to team with external organizations who can help bridge this gap. 

Originally Posted on Wed, Jun 17, 2015 @ 09:23 AM

Photo by Samantha Sophia on Unsplash

No responses yet

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.