Business intelligence (BI) boils down to one thing: helping people make better choices so they can do their jobs better. For 20 years, BI has done this through standardized reports and dashboards, best-practice scorecards and KPIs.
But as more people rely on more kinds of information for more decisions, BI’s tried-and-true technologies and techniques will struggle to deal with the diversity of daily decision-making. As growing data volume translates into growing data variety, like unstructured documents and Internet content, the problem just gets worse. This is BI’s next grand challenge: diversity at scale.
The BI industry knows this and is taking advantage of hardware advances and new deployment approaches. Over the next year the three big trends in BI will be: fast, agile and mobile.
Fast: BI Be Nimble, BI Be Quick
We’re so familiar Moore’s law that we forget to marvel at its effects. Multi-core, multi-chip 64-bit systems usher in the age of commodity supercomputing. The kind of number-crunching that could only be done on an elite multi-million dollar machine five years ago can now be done with an off-the-shelf server. And the relentless advance of the price-performance curve means that not only servers, but data warehouse appliances, and even mobile devices will continue to get faster and cheaper. That means faster answers on more data. But faster is only part of the story. Like a plane accelerating for takeoff, at a certain speed something entirely new happens.
Agile: Side-To-Side BI
There’s a lot of talk about what Agile BI is, what it isn't, and where it fits. Some focus on agility in deployment – less requirements development, less documentation. Others focus on agility in the user experience – more self-service exploration, fewer requests for custom reports stacking up in the BI backlog. Which is it? Go back to the grand challenge and you can see that it’s both. When you bring the two kinds of agility together BI takes flight.
Start with the user experience. One of our customers has a great phrase for agility in the user interface. She calls it side-to-side BI. Drilling up and down in reports and dashboards is good, but the real magic comes in drilling over, around, and between. Agility for people using BI is being able to answer the in-the-moment questions inspired by reports and dashboards. For example, when the report says that warranty claims on the company’s top-selling product went up 45% last month, a whole host of new questions come to mind. Is this normal? Across packaging options? By customer segment? What are customers saying on the blogs? No one knew these questions would be asked or that they would be so important. Agility is being able to answer unanticipated questions as they arise.
But to build a system that gives users such freedom, IT will need agility too. The information that answers the warranty manager’s questions is spread across product databases, marketing datamarts, and blog entries. The meticulous up-front modeling of traditional BI is too labor-intensive for diverse data like this. And the old-world approach of making incoming data conform to the model leaves behind details in the sources that may explain the spike in claims. Then the data changes – new websites or Facebook applications, supplier data, or manufacturing quality documents become important to the investigation and have to be added into the analysis. Agility is being able to bring together diverse data and content whether inside or outside the company, and then rapidly respond to changes in that data or from additional sources.
The blistering performance of in-memory databases is the key to agility. With newly abundant processing power, BI shifts from up-front modeling to model-as-you-go, allowing users to organize information in-the-moment and allowing IT to add new data as needed.
Mobile: In-The-Moment, On-The-Go
Daily decision-making doesn’t just happen at a desk. Much of it is out in the world where only mobile devices can go. The basic promise of mobile reports and dashboards is straightforward – look up facts and figures. This is a good start. But the bigger promise is in exploration and discovery on mobile devices.
Imagine a dealership manager for an auto manufacturer going on site visits to see how things are going. The warranty department just found out the bad news about warranty claims on the top-selling model. Which dealerships should our manager spend more time with? What kinds of questions should she ask while she’s there? Combining superior access to content and data with on-the-ground observation could be the difference between getting the jump on an emerging problem and letting it linger until it becomes an expensive, embarrassing problem.
The warhorses of BI, reports, dashboards and KPI scorecards will never go away. They provide real value by answering questions the company knew to ask in areas that it already understands. But accommodating diversity at scale is really another way of saying BI’s new grand challenge is to deal with the rest of the world the way it really is – full of disparate, changing information and different, varying questions. BI that’s fast, agile, and mobile is the key to tackling this challenge.
About the author
As Endeca’s Chief Strategist, Paul leads the company’s efforts to evangelize the benefits of information visibility and its impact on competitive advantage. Before joining Endeca, he spent six years as an analyst at Forrester Research, focusing on search technology and experience design. Paul's investigations into search included deep dives into information-retrieval theories and technologies; examining these through the lens of user experience showed which ones actually help people discover what they're looking for. Paul published numerous reports on these topics and helped hundreds of executives at Global 2500 firms apply research to solve real business problems. Prior to Forrester, Paul worked at Strategic Interactive Group (now Digitas), where he helped clients set business objectives for online initiatives and translate those goals into technical requirements for development. Paul holds a B.A. in English from Wake Forest University.