The report design working group sat around the table, sifting through the draft strategic performance report to suggest how to make it more useful. Measure by measure they chatted and suggested and critiqued and debated: “this one would look better if it was a bar chart”, “yeah, I like the three-dimensional bar charts”, “we should add another line to this chart because it would be interesting to show”, “it’s pretty easy to get Excel to turn this one into a stacked bar chart, that way we could get more information onto it”. Then someone asked: “hang on, what questions are we trying to answer with these measures anyway?”, and there was dead silence.
Are you using this kind of performance information?
Performance reports are most commonly filled with a combination of information like the following:
- tables of comparisons of this month with last month, this month with the same month last year, year to date with target
- the default bar charts that Microsoft Excel formats for you
- fancy three-dimensional, stacked bar charts
- other charts that are crowded with information that looks visually exciting and at best, might be interesting
- occasionally some short term trends, consisting only of 5 or 6 or barely a few more points of data
- commentary about project or initiative progress or milestone completion
How many of these kinds of information are in your performance reports? How much of that information is read, valued, validly interpreted, understood and applied to inform decisions? How aligned to your organisation’s strategic, tactical or operational priorities is this information?
What’s wrong with this kind of performance information?
If you answered these three questions with “quite a lot”, “not very much of it” and “not very well”, then you’d be fairly normal. Most organisations’ performance reports are created with only a basic awareness of good business statistics and even less of an awareness of the business questions the report should answer.
Most of the information provided in these reports is in the form of “limited comparisons”, to borrow a phrase from Donald Wheeler (1). Limited comparisons can’t really answer any business question, because they are not a representative picture of the performance results they monitor. This is due to the fact that there is always natural variation from month to month, week to week, day to day, and 2 points of data can never tell you what amount of variation is normal versus abnormal.
What makes matters worse, is that often it seems this information is designed this way purely because it always has been. Rarely is the design of the information questioned or challenged. And rarer still, is the design of the information reviewed in the context of the business questions it must answer. Information design, particularly the visual design of quantitative information (Edward Tufte has written some amazing books on this very topic – see references 2, 3 and 4), is a real body of knowledge, linking statistical theory with cognitive theory to provide insights into how we can make information more useful and usable in decision making.
What are the right business questions to answer?
What is business performance management really about? Ultimately it’s about business success, and providing the information to make the decisions that increase that success, now and into the future. Performance management is about three specific things. Firstly, it’s about monitoring your business’ actual progress toward the outcomes (and targets) implied by your business strategy. If one of those outcomes is to increase customer loyalty, then it’s about monitoring how much customer loyalty you have as time goes by (such as the average number of orders per customer per quarter), and comparing this actual level of customer loyalty with the targeted level (say 20 orders per customer per quarter).
Secondly, business performance management is about knowing which of your initiatives or projects are working and not working in making those outcomes happen. If you have an initiative around developing a customer relationship management system to improve customer loyalty, then you would expect to see that customer loyalty increases the more the customer relationship management system is implemented and used. If you don’t see a change in customer loyalty despite implement customer relationship management, then how can you say the system was working? You can’t.
Thirdly, business performance management is about knowing why those things are working or not working so you can choose better things to do, or fix the things you are doing. Perhaps the customer relationship management system isn’t impacting customer loyalty because customers are already happy with their relationships with you, but just feel your products or services aren’t as relevant as they expected.
This description of business performance management suggests several specific questions are important in managing a business so it’s success improves.
Table 1: examples of excellent questions for business performance management
|Have we achieved our target?
||We monitor business performance so we can know when we are actually performing at the level we need to, or want to, perform at. Targets are the description of the “need to” or “want to”.
|Are we progressing toward our target?
||Rather than just waiting to the end of the year, or the date we wanted to achieve the target by, monitoring continuously throughout that timeframe gives us more power to influence the end result.
|Are their any unintended consequences of our actions?
||Chaos theory, the butterfly effect, and system thinking all tell us that there will always be some kind of flow-on effect from our actions. These flow-on effects can be anywhere from small and insignificant, to shockingly dramatic.
|Why are we getting the results we are getting?
||This is a question that is seeking information about reasons. Of all the possible reasons that you are getting a particular performance result, which reasons are the main ones, those that have most of the impact? If you are getting a good result, then knowing these reasons helps you confirm what to celebrate and keep doing more of. If you are getting a bad result, then knowing these reasons helps you modify your course of action to turn the result around.
|What is likely to happen in the future?
||information in business. While no information can really do the crystal ball thing, a really good understanding of drivers (or lead indicators) can certainly give some great clues about likely future results. Thus, you can prepare for the most likely outcomes, before they happen.
Design your information to answer your business questions
The types of information needed to answer these business questions are different for each question. And unless that information is designed with the question in mind, it’s likely that you won’t be able to answer the question, or if you do, it will suffer the risks of misjudgment.