The most successful companies today are the ones who have acted upon their data assets by leveraging advanced analytics and big data technologies. If you think of Apple, Netflix, Google or FaceBook, the one thing they all have in common (in addition to being "tech" companies) is that they have highly evolved analytics strategies and practices. One can safely say that all of these companies are "data" companies, and not device or video rental or search or social media companies. Add to this list, a company like Domino's Pizza. I had the opportunity to visit their campus yesterday and was amazed by how data driven the business is. In fact, Domino's proudly announces in their lobby that, outside of Amazon and Google, they are the largest consumer of big data analytics. So who exactly are the actual users of all of the big data and analytics? Today the largest users of big data in business are the folks in marketing. They need to leverage Hadoop for everything from sentiment analysis to real time product recommendations.
However, in my mind, the biggest use case for Hadoop is still yet to come. A company called Camgian recently released a product called Egburt which is simply a "black box" (actually it is white) to collect and monitor sensor data from real time activities or quantities such as foot traffic, freezer temperatures, structural stresses and so on. This is the brave new world of IoT! On this blog, we have discussed previously similar black boxes that can be put on vehicles to monitor real time process data from various control units. Today's automobile has literally thousands of control units (we could call them meta-sensors).
Auto manufacturers say that just one vehicle can potentially generate 25 GB of data per hour! Most of this data conveys nothing more than what could be called "normal operating conditions" of a vehicle. For example, temperature, oxygen levels, pressure, voltage and a myriad other physical par
ameters which mainly concern only the engineers who develop the engines and other hardware. That too, only in the initial design and prototype stages of development because this data gives them information about potential inefficiencies or perhaps malfunctions which can be fixed in the next design iteration.
Part of the reason why very few people were interested in this mass of streaming data is that no one actually collected this before. I mean, some data was collected, perhaps averaged and perhaps extrapolated, but the whole mass of it was too much to store and way too much for any analytical tool. However the arrival of technologies like Hadoop and Apache Spark are likely to change this scenario. Now, storage is no longer an issue, and very soon computation will also be a non-issue. But big data companies are the first to state that "you do not have a big data use case, simply because you generate a lot of data"!
So what can you actually do with this streaming data? Three things come to mind. The first was already mentioned - use this to establish what would be considered "normal" operating conditions or baseline performances of complex systems. The second is to use the trends in this baseline data to predict potential problems. The final frontier, so to speak, is to then let the system decide the best way to fix this problem. Tesla, apparently already did this recently to overcome a potential recall problem by letting their cars automatically adjust suspension settings.
Arthur C Clarke said that one day computers will not only compose music, but they will also learn to enjoy it! IoT and analytics will very soon allow our machines to not only identify their problems, but also fix them without us even knowing about it. Exicting or scary?
Download a case study to see simpler applications of analytics in manufacturing.