This column is a follow-up to Kirby Lunger's excellent Dashboard Insight article of October 11, 2007: A Performance Manager Primer, and provides an introduction to the approach used by OpenBI to detail performance requirements in our BI engagements.
For OpenBI, the point of departure for performance management is an understanding of our customer's business strategy. Strategy has been simply defined by Michael Porter as that set of unique activities a company deploys to achieve its unique self concept. Strategic positioning for differentiation and competitive advantage results when a company effectively performs a different set of activities than its competitors, or conducts similar activities in different ways. Successful competitive strategy is about being different, about choosing different sets of activities to deliver unique value, and about making trade-offs in activities to achieve optimal outcomes.
The connection between strategy and intelligence has been well-chronicled by Robert Kaplan and David Norton, innovators of The Balanced Scorecard. K&N take Porter's strategy to the next level of measurable detail, viewing strategic activities as positioned in a series of hypotheses or theories designed to move the organization from its current state to a desirable but uncertain future state. These hypotheses link strategic activities to explicitly measurable outcomes as cause and effect. Three classes of variables are included in hypotheses linkages: factors, lead indicators, and lag indicators. A company has control over factors, which can include, for example, what markets to be in, types of customers to target, what to sell, and the media to sell through. Lead indicators are measures of performance that the company envisions sitting between factors and ultimate success in the marketplace, and include variables such as web click-throughs, response to marketing experiments, and retention campaigns. Lag indicators measure company success in the marketplace with focus on financial metrics such as revenue, profit, growth, and loss ratio. Key performance indicators are lead and lag benchmarks established by the business and BI team and used to calibrate performance. Linkage hypotheses relating cause and effect take the form of: If A (factor), then B (lead indicator) or the more/less of A (lead indicator), the more/less of B (lag indicator). A strategic linkage starts with the relationship of factors to lead indicators and is followed in turn by lead to lag indicators. A company’s composite strategy is the aggregate of these hypotheses/linkages. Support of the company’s efforts to assess and improve the effectiveness of this strategic roadmap is thus a defining role of BI.
A major benefit of Kaplan and Norton’s approach is that of operationalizing strategy --moving from vague and conceptual to explicit and measurable. At the point of explicit and measurable, there is a confluence of strategy and BI. The primary role of BI for both the evaluation of operational effectiveness and strategy execution is performance measurement (PM). Before addressing the usual BI functions of data integration, query/reporting, and analytics, the BI/PM analyst must work with business to make explicit and measurable the linkages underpinning strategy, precisely specifying factors/indicators, relationships and outcomes. A more subtle, but equally important, task of PM BI analysts is one of promoting methods to ensure that lead and lag indicators are indeed related as cause and effect.
OpenBI distinguishes a second supporting role for BI, performance management, in addition to the measurement noted above. Both PM’s include the role of BI detailed earlier to operationalize strategy, build supporting data stores, and analyze intelligence data. Performance measurement stops there, however, content to passively evaluate the top-down strategy provided by the business. From this perspective, measurement is pretty much equivalent to traditional BI -- merely reporting the results of performance with no outward focus. Performance management, on the other hand, is actively engaged with the business, measuring like traditional BI, but adapting the strategy to the measurements in a feedback loop. In addition, the better performance management solutions provide focus to test and improve performance. Not satisfied with mere reporting, performance management assumes more of a bottom-up, experimental focus, offering advice on how to tinker with strategy, using its BI findings, the experimental method, and wisdom accumulated from evidence to steer the company in new directions.
Rather than just passively observing the results of a new initiative, the BI/PM team, for example, might propose a randomized trial with treatment and control groups to test strategic alternatives, a superior design to observation only. An illustration would be a retail cataloger that mails different versions of its primary catalog to randomly selected cohorts and a control group, then measures subsequent purchasing behavior. With such a design, the organization can be confident that differences noted in the buying performance of the disparate groups are due to the catalog versions they received – that the catalog versions in fact caused the differences in buying patterns -- and make strategic accommodations accordingly. Leading intelligence companies now routinely include experiments as a means of testing and improving their strategy.
The following table shows a snippet from a hypothetical BI/PM requirements study for an insurance company that details factors, lead, and lag indicators for the underwriting function. The content would have been formulated through a series of senior management meetings led by OpenBI partners. Strategic linkages, presumed cause-effect relationships between the factors and indicators, are detailed in another section of the requirements report. For a company committed to experimental management of its processes, this type of information provides guidance for measurement as well as a means to test and improve the strategic underwriting process.
Factors
|
Factor |
Definition |
|
Underwriting Rule Engine |
The version of the underwriting engine which includes all rules used to quantify an applicant’s risk |
|
Application Questions |
The version of the insurance application including versioned questions which the applicant completes and submits in order to obtain an underwriting decision and rate quote. |
|
Application Answers |
The applicant’s responses on the application |
|
Applicant Source |
The lead source of the applicant |
|
Applicant Demographics |
Gender, Age, Geography, etc. |
Leading Indicators
|
KPI |
Definition |
Threshold(s) |
|
Preferred Applicant % |
% of applicants who receive a preferred quote |
> 60% |
|
Non-Preferred Applicant % |
% of applicants who receive a non-preferred quote |
< 30% |
|
Non-acceptable Applicant % |
% of applicants who are declined for all plans |
< 10% |
|
Enrollment % |
% of applicants who receive and accept an offer and complete the enrollment process |
> 75% |
Lagging Indicators
|
KPI |
Definition |
Threshold(s) |
|
Earned Premium |
Collected premium for already past coverage days |
|
|
Incurred Claims |
$ of incurred claims |
|
|
Loss Ratio |
Incurred Claims / Earned Premium |
<65% |
|
Fraud Frequency |
% of members who file fraudulent claims and/or not truthful on their application |
|
This brief article provides an introduction to an approach for mapping business strategy to the explicit measurement of performance that provides the basis for BI and business performance management. The process of working with the senior leadership team to divine these inputs may be challenging and even arduous, but, done effectively, will produce results that can start to align the business around a clearly-articulated focus on performance. If, in addition, the engagement succeeds in introducing or refining a focus toward the experimental method as another improvement tool, so much the better.
References:
1. Michael E. Porter. What is Strategy? Harvard Business Review. November-December 1996.
2. Robert S. Kaplan and David P. Norton. The Strategy-Focused Organization: How Balance Scorecard Companies Thrive in the New Business Environment. Harvard Business School Press, 2001.
3. Kirby Lunger. A Performance Manager Primer – Why You Need More Than a Dashboard to Manage Your Strategy. Dashboard Insight. October 11, 2007.