How to use mutual information for key performance indicators [VIDEO]
Wikipedia derides Key Performance Indicators (KPI) as "industry jargon" for any type of performance measurement. You may wonder why.
The overall goal of setting up key performance indicators is to have a systematic, measurable and objective target to shoot for, while monitoring the success of any business activity. For example, a sales department should have KPIs like average sales price, sales conversion time, average cost of sales and sales efficiency - to name a few. A logistics or transportation department may have cost per unit shipped, vehicle time utilization, on-time delivery etc as their key performance metrics. So now it may become apparent to you why the wiki plays down on KPIs - too many channels of measurement and not enough information on how they rank relative to each other. It may lead to the "we have all this data, but what next" question.
Two problems with out-of-the-box Key Performance Indicators
The KPI Library has hundreds (if not thousands) of key performance indicators sliced along industry verticals and process horizontals in current business use. This adds to more uncertainty about which KPIs are best for your particular business.
The Advanced Performance Institute lays out clearly one of the potential problems with standard KPIs: data overload. They correctly state that businesses sometimes collect too much data and then find themselves at a loss on what to do next. "Instead of clearly identifying the information needs and then carefully designing the most appropriate indicators to assess performance, we often observe what we have termed the ‘ICE’ approach:
- Identify everything that is easy to measure and count
- Collect and report the data on everything that is easy to measure and count
- End up scratching your head thinking “What the heck are we going to do with all this performance data stuff?"
When you try to track too many KPIs, there is the issue of redundancy. Many of these parameters could be strongly correlated and thus we would be better off keeping a single "hub" variable and let go of the "spoke" variables. This will reduce the data overload and if you want to perform analytics such as regression modeling, removing correlated variables is actually a required step.
The second problem with generic, one size fits all KPIs is that departments tend to look at them in isolation. For example, consider what would happen to a business if the objective of Technical Support is to ''increase customer satisfaction levels to 80% by year end'', whereas the call center has decided one of their KPIs is to handle 30 calls per hour. Are these two goals mutually compatible?
This is where mutual information based analysis will work very well to develop a shortlist of key performance indicators. There are several articles that explain the technical details behind mutual information analysis and we will not discuss them here. The first problem is eliminated by ranking all the selected KPIs according to their order of information content. The second problem (of interaction) can be reduced by exploring the main relationships each of the top KPIs have with each other and judiciously selecting the ones which do not have this compatibility problem.
Mutual information based analysis will identify all the KPIs which carry the most relevance to a specified business objective (also called a Target variable). This is the first step in identifying the KPI shortlist. Once this is done, the next step is to cycle through the shortlisted KPIs to examine if any of these are incompatible.
The following video demonstrates how one can develop a list of the most appropriate indicators of performance and make sure that there are no compatibility issues. The video shows how to use KeyConnect, an online SaaS tool for performing this sort of analysis.
1. Collect all the KPIs of interest. In this case, we have 9 different transportation related KPIs loaded onto a .csv file
2. Select your performance objective. In this case, our objective is to keep the cost per unit (CPU) shipped as low as possible and therefore we select the CPU as the "Target Variable"
3. KeyConnect will show three separate chart outputs. The first chart on the left is a bar chart showing the ranking of 4 KPIs which have a significant impact on the objective (CPU). The chart in the middle shows how each of these four are related to the CPU. A thicker yellow line indicates a stronger influence on CPU. The chart on the right is a scatter plot which will allow you to drill down into the relationship between CPU and any of the available KPIs.
4. You can save the analysis and return to it later and perform a more stringent analysis, for example by setting the filter threshold at "Low" or even changing the objective or target variable.
5. You now have the four main key performance indicators which significantly influence your business objective, down from 8. That is a 50% reduction of your data overload.