THE NEED FOR A NEW INFORMATION-DELIVERY APPROACH
Thanks to the highly communicative global economy, sustaining market advantage and avoiding commoditization of a business's innovative excellence and competitive edges are only possible through embracing accelerated change. This highly adaptive environment requires an extremely responsive information infrastructure that can embrace the high degree of change required to excel in this new global economy. The current genre of business intelligence solutions are applications hard-wired to yesterday’s problems, which is insufficient for today’s accelerated, highly adaptable business environment.
Every fifteen to twenty years, a new approach surfaces for delivering information to those accountable for deriving value for an enterprise. In the mid to late nineties, the era of Executive Information Systems (EIS) was eclipsed by emerging Business Intelligence solutions. Today we are witnessing such a transformation once again in response to the limitations of BI and the potential of new capabilities in delivering actionable, trustworthy just-in-time information that helps people drive business. These new capabilities, which I call “Post-Discovery Intelligence,” once again provide a radically different approach to delivering information – this time by focusing on eliminating the limits on the sources and types of information that can be accessed and at the same time providing a much more flexible environment for taking action on the valuable information that is retrieved.
A BRIEF HISTORY OF ANALYTIC INFORMATION DELIVERY
Figure 1: Mass Commoditization of Previous Analytic Information Delivery Solutions
The ability to provide analytic information to those accountable for creating value for the organization was a primary focus when mass availability of computing power became reality. Starting with the management science era that began in the 1970s, the focus has changed little: it was and continues to be the delivery of actionable information that assists senior management in steering the organization by quickly identifying patterns that suggest an opportunity for improvement and bumps in the road that demand their attention.
What has changed recently are four very important items that are driving the current post-BI era:
- The pervasive availability of remote, secured computing power
- The continuous growth of the data deluge, accompanied by new capabilities for locating all the appropriate information in the deluge
- The rapid acceleration of the pace of business, largely impacted by the mass adoption of always-on media (Twitter, IM, SMS, MMS, RSS, blogs, digg, livejournal, del.icio.us, etc.)
- The desire for more in-process intelligence that responds in near real-time
DRIVERS FOR ADOPTING A POST-DISCOVERY SOLUTION
As so often happens, a new solution is in large measure a reaction to what was wrong with the solution it is replacing. It is interesting to note that while we have spent the past fifteen or so years focused on enhancing the stack used to derive intelligence from information, these efforts have created new dilemmas, some of which can only be resolved with a significantly new approach.
|We wanted to solve:
||But also introduced these issues:
|To ensure information was complete, we specialized the components used to derive intelligence from information in diverse repositories
||The stack has become increasingly complex and rigid as we add layers intended to resolve data differences and present uniformity (e.g., Master Data Management)
|To provide insight without compromising the performance of operational systems, we designed complex underlying structures that avoid performance fluctuations
||The platform constructed for insulating operational systems suffers from some of the symptoms that resulted in building the warehouse due to the sheer growth of information volumes, information consumers, sources and transformation complexities, all of which were never envisioned when the initial models for data warehousing were devised
|To enable reporting self-service, we provided a data model that could support multiple reports
||The data model used for self-service is so complex that only those with specific training can use it, and it often doesn’t reflect the important time critical issues that surface which demand immediate attention
|To improve the value of reports, we wanted to simplify the reporting and analytic environment for subjects that our stakeholders cared about
||We hardwired data marts along the reporting dimensions of what was important when the data marts were built. Because priorities shift more quickly in this highly communicative global marketplace, the value of these solutions diminishes rapidly
|To improve insight, we wanted to provide context about the performance systems (Business Intelligence, Planning and Forecasting systems, etc.)
||The knowledge management systems grew up in different organizations, used different technologies and often different organizational schemes, so syncing the systems is complex, slow and compromised
|To address stakeholder demand that business information be as easy to get as information on the internet via Google, we integrated knowledge management systems with our performance systems
||Each alternative to solving this problem was too much of a compromise because BI and Enterprise Search technologies were too incompatible
|The world sped up, and our stakeholders clamored for information delivery to be much more responsive
||We started down the path of operational BI, but the data model, which started the whole journey, lacks the flexibility to be as responsive as stakeholders are demanding
ENTER THE POST-DISCOVERY SOLUTION
Post-discovery is a very different approach to delivering actionable, trustworthy information to stakeholders – a way that provides the responsiveness clamored for by stakeholders and at the same time avoids some of the pitfalls that we introduced during the past fifteen years as the sheer volume and half-life of information has grown past our wildest imaginations.
The Post-Discovery approach for building intelligent applications is very different from the traditional approach. In the Post-Discovery approach of constructing intelligent applications, the underlying data model serves not as the glue that holds together the storage, presentation and transformation of information but rather as a reference model. The benefits of this approach are clear. Just as production applications have benefitted tremendously from decoupling the components of an application (i.e., Service Oriented Applications), so too do intelligent applications benefit from a similar decoupling – in this case a decoupling from the hard-wired information model.
The differences between BI and Post-Discovery include these approaches to key components:
||How this is handled in a Pre-Discovery solution
||How this is handled in a Post-Discovery solution
|Underlying Data Model
||Pre-designed and pre-defined common dimensional model represented in user interface, ETL layer, database layer, business rules layer, custom navigation layer
||Information Catalog is loosely connected to the index into information, which is navigated using named pairs, a navigation scheme similar to that used by Google, Yahoo and Bing.
||Specific tool suite that is intelligent about underlying data model
||Agile user interface that manages a loosely coupled catalog containing information about the data flows and transformations
|Extract, Transform & Load (ETL)
||Rules-based engine or code conversion engine, governed by workflow in many cases. Information must be reshaped to conform to the data model.
||Workflow engine that is aware of the loosely coupled catalog.
||Integrated into the BI platform, ETL platform, custom integration layers and database platform(s)
||Integrated into the user interface platform and workflow platform
||Difficult integration through a custom interface
||Slice and Dice
The Post-Discovery process is partly about tools but is much more about adopting a new process that reintroduces the agility and relevance your intelligent applications had when you first constructed them. Because of the high degree of relevance, agility and measured participation in incremental value creation, it also ensures that stakeholders’ level of enthusiasm remains high.
WHY GO THROUGH THE EFFORT OF ADOPTING A NEW MODEL?
There is a major change going on in the marketplace. Because news circumnavigates the globe so quickly, and innovative ideas commoditize rather quickly thanks to our over-communicative global economy, your business requires a keen ability to meet the demands of an accelerated rate of change just to sustain your current market positioning. Unfortunately, many of your information resources, required to chart the course for this accelerated rate of change, haven’t kept pace. This is largely due to the time-consuming processes used to ingest new information and the high degree of complexity in the information models and transformation processes used to ingest current information.
Just as troubling is a recent study by IBM, Accenture and InfoSight Partners1 that shows most people know the information they are using for charting a course for their organization will be either wrong (50% of the time) or useless (42% of the time), and 52% have deemed their information untrustworthy.
The old practice of ensuring all information is represented in the information model and assuring organizational stakeholders that “it’s like Prego, it’s in there” just doesn’t surface welcome reactions any longer. A better means of providing focused, relevant, actionable information is mandatory. This new method, which can be thought of as Post-Discovery in contrast to the inflexible nature of pre-discovery, shifts the focus of information delivery to agility.