Article written by Claudia Imhoff, Ph.D., Intelligent Solutions.
In today’s dynamic environment, organizations are faced with new and
increasingly critical decisions which can affect their very survival.
Decision makers, in turn, have placed more and more demands on their
Business Intelligence (BI) environments. Their demands are increasing
the pressure on IT to deliver the right data at the right time, and faster
than ever before.
“The right data” means well-documented, reliable, and consistent data.
“The right time” means delivering the appropriate data in a timeframe
appropriate for the particular decision. That is, delivery with enough lead
time to allow the effect(s) of the decision(s) to make a difference.
“The appropriate data” means the combination of historical data found
in data warehouses and data marts, low latency data found in an
operational data store, and real time data from operational systems.
Bringing the appropriate data together requires sophisticated data
integration infrastructures that go way beyond simple data
combinations and displays. So, data integration requires a mixture of
techniques and technologies – each specific to the type of decisionmaking
In this paper, the two most popular techniques for data integration
(virtual data federation and physical data consolidation) are described,
and the data integration technologies that support each technique (EII
and ETL) are reviewed. Through eight BI examples, advice is offered to
help implementers determine when to use each technique and
A Strategic Architecture for BI
Today’s enterprises must be fueled by good decisions – about their
customers, products, employees, partners, and more. Business
Intelligence (BI) has long helped the company’s decision makers by
supplying them with reliable versions of historical data – snapshots used
to determine trends, patterns, comparisons over time, etc.
Strategic decisions are certainly critical to the ongoing well being of the
organization, but are not the only type of decisions being made.
Today’s BI architecture must also support a myriad of operational
decisions, driven by today’s dynamic business environment.
The public domain, logical architecture that I advocate for BI is the
Corporate Information Factory1 (CIF). Over the years, the CIF has
earned its stripes as the most reliable and most implemented
architecture for sustainable BI.
Figure 1: The Corporate Information Factory
Several CIF components – the data warehouse, data marts, and
operational data store – have remained fairly stable over the years. The
data warehouse is still the main source of generic, integrated, historical
snapshots of data. The operational data store is used to store low latency or near-real time integrated data. The specialized analytical
data marts are dependent on the data warehouse or operational data
store for their supply of data and contain the analytical applications.
Each serves a specific purpose in handling planned and unplanned
queries and reports.
Although these components have endured, the way we create them
has changed dramatically. While there is no question of the value of
historical snapshots, the time latency involved in the preparation of the
data makes them unsuitable for intraday or operational decision
making. So, the CIF evolved to support not only traditional, strategic BI,
but also tactical and operational BI.
Data Integration Techniques & Technologies
Today, BI implementers are fortunate to have two data integration
techniques available to help them create world class environments – CIF
or otherwise. These are Physical Data Consolidation and Virtual Data
- Physical Data Consolidation – This technique uses processes that
capture, cleanse, integrate, transform, and load data into a target
data store. Typically data is consolidated using extraction,
transformation, and load (ETL) technologies, which obtain data
from operational data sources, transform it to the corporate
standard, and load it into physical data stores. Informatica’s
PowerCenter is an example of an ETL product.
- Virtual Data Federation – This technique uses processes that provide
a real-time integrated view of disparate data types from multiple
sources, providing a universal data access layer. Data federation
uses enterprise information integration (EII) technologies to create
virtual stores of data from data warehouses, marts, operational
data stores, and operational systems. Composite Software’s
Information Server is an example of an EII product.
The biggest question for implementers is not whether to use these
techniques, but when to use them. Each has its place within a data
integration infrastructure but it is not always clear when to use one or the
other. To help implementers make the “when” decision, eight examples
are presented below, including the rationale, the benefits, and the
drawbacks of choosing one over the other.