There’s no doubt that dashboards are a valuable tool in an organization’s technology arsenal, but many continue to grapple with challenges, such as the types of data to include, where that data resides and how to incorporate data in different formats. In an effort to make more informed business decisions, many departments and individual line-of-business users have turned to Business Intelligence (BI) dashboards as a way to present data for both reporting and forecasting purposes and to address increasing business issues.
Unfortunately, with failure rates hovering between 50% and 70%, according to some studies, and business users continually looking for new tools to help them easily get at the data they need, BI alone is not always the answer. As a result, a new approach has emerged that analysts refer to as Report Analytics. These tools model, aggregate and transform diverse data from any number of existing reports and business documents in various formats throughout the organization. This makes it easier and far more cost effective for users to access, extract and analyze data without having to invest in new reporting solutions.
It’s well known that BI software can provide advanced reporting and predictive analysis, but for many day-to-day reporting tasks (especially those that are repetitive) its sophisticated functionality is overly complex. For tasks like these, business users and self-service “data consumers” may find the complexity of BI a deterrent and as a result, often rely on the IT department to generate custom reports. This not only creates more work for an already over-taxed IT group, but also introduces delays and frustration for the users who require accurate and timely information in order to perform their job effectively. The bottom line is that in order to have the best information at hand, organizations need BI and report-based dashboards that are capable of integrating that data from a multitude of systems.
The underlying issue is that business users often have all the information they need; however, that information resides in existing reports and business documents scattered throughout the organization. In fact, much of the critical data often exists outside of the BI system. Since they have no easy way to dynamically organize, integrate and analyze the intelligence trapped in these static documents, they are often left with less than ideal options. For instance, business users can rely on predefined BI reports from a data warehouse or enlist the expertise of the IT department to program custom reports. The ideal scenario – having a BI dashboard that includes document and report-based information with relevant data stored in databases and data warehouses – often eludes many organizations.
The Challenges of BI Today
Organizations of nearly all sizes generate thousands of reports monthly and many are from ERP systems, mainframes and more, and are often hundreds of pages long and in a variety of formats. Despite the investment in those systems and the strategic nature of the reports, they are largely unusable without investing time and money into burdensome, manual processes. For dashboards in particular, users need to determine how to automate the process of getting the data from the source system into the dashboard. Often, the data does not reside in a relational database, but in a semi-structured format like EDI files, log files, PDF files and mainframe print spools. While custom programming can sometimes pull the right data, it isn’t the best use of budget and resources on an ongoing basis. Report Analytics software leverages an organization’s existing reports and reporting processes, provides consumers with a self-service dashboard environment. This allows them to extract the relevant intelligence from any combination of these existing documents themselves and transform that information into dynamic, interactive reports for analysis and visualization. Whether the reports or business documents originate inside an enterprise or from external sources like customers or suppliers, Report Analytics allows business users to create, distribute and publish these reports without time delays or involving IT. And let’s face it, with increasing cost pressures and decreasing revenues, anything that bolsters productivity is critical in today’s real-time business environment.
In most organizations, the volume of data is enormous, creating a challenge that lies not in amassing more data, but rather in integrating and using the meaningful data that already exists. Business users are often forced to spend valuable time aggregating information from various sources. And that’s not all. Once they have access, they still need to manipulate the data in familiar programs like Excel in order to present the information in the right format, to the right people, instead of focusing on the more value-added activity of analyzing and acting on the data itself. According to a recent Ventana Research study, this creates an issue for many businesses. The study found that leveraging reports from BI systems is important to 57 percent of organizations, and getting to the data from source ERP, CRM and other applications is important to 71 percent of respondents.
To further complicate matters, existing reports and business documents come from a variety of sources, including internal transaction systems that generate canned ERP, HR or CRM reports, external customer or supplier systems, and personal productivity tools such as Excel or BI systems. The challenge stems from the fact that these reports come in multiple formats: mainframe green bar, text, ASCII, PDF, HTML, spreadsheet, log files and semi-structured documents that are stored in content management systems. All of these sources and formats of existing reports creates a major challenge for business users that need to make timely, informed business decisions based on available data.
To make matters worse, the “simple” ideal of having the right data in the right place is the type of data generated by the average organization. BI systems are largely restricted to handling structured data. Today’s BI dashboards need to incorporate not just structured data, but also diverse data that presents in semi-structured and loosely structured formats. Report Analytics has the ability to handle all of these diverse data types, and does so easily, and cost effectively.
Making Data Meaningful
Today, the enormous amount of data has made it difficult to parse and analyze, which means that the resulting reports are sub-optimal at best. The format of the report also comes with its own set of unique problems. ERP, HR and CRM systems deliver static reports, oftentimes in text or PDF format, which are inflexible and cannot be integrated with data from other reports. This does little to foster understanding, analysis, or decision making and often does not include the unstructured, semi-structured, or externally sourced data that is required to make information meaningful. As a result, most organizations spend a great deal of money and time consolidating and mapping this data into a data warehouse, data mart or other operational data store. Worse, an organization simply abandons the idea of leveraging that data and operates by intuition rather than hard data.
Report Analytics began to take root as a way to address this longstanding business challenge. Functioning as the “missing link” in the broader BI reporting arena, Report Analytics captures structured and semi‐structured data from virtually any existing document. It also enables faster and deeper visibility into the business and better, more informed business decisions.
Report Analytics & BI - Not an Either/Or Proposition
Report Analytics may sound like it overlaps with Business Analytics or Business Intelligence, but l they are actually complementary, yet key differences exist between the two. For instance, in BI deployments a few staff members, largely from IT, are charged with managing the data and creating both content and reports for business users. So while BI systems are an invaluable asset for many - allowing them to discern patterns in customer behavior and align the business behind a common goal - it may not always be the best or only tool for generating reports. Business intelligence dashboard and decision support requirements are similar to business intelligence reporting requirements. Depending upon the complexity of the decision, data may be required from historical data storage, such as data warehouses, and on-line applications. It is essential that an experienced business intelligence analyst, data analyst, and the business team develop requirements early in the project.
Of the IT staff questioned in the Computing study, 31 percent said their department was required to produce up to 100 reports a month while 16 percent generated up to 500. Even worse, four percent estimated that they were required to produce more than 1,000 reports every month. The time that IT staff spent on producing these reports varied, with half taking a day or more and 10 percent estimating an average time of weeks or months. With such a limited group responsible for developing reports for the entire organization, back-logs undoubtedly will occur. In other words, it requires lots of IT support to make information usable.
As a result, while IT is off programming the custom reports, the business requirements change which means the data that consumers receive in their custom report is once again incapable of helping them solve their particular business problem. This leads data consumers to the ever-popular BI “workaround” – Excel. Business users dump the data they need into Excel, import additional information from transactional systems or other sources that are not in the data warehouse, and then perform analytics. However, this cumbersome manual process is necessary as there is often little analytic value because of the missing and incomplete data. While it solves the business user’s immediate need – getting the data they need to answer specific questions – it is fraught with peril. On a tactical level, moving data from a trusted system to the unsecure environment of Excel also introduces the possibility of compromising data accuracy due to simple human error. On a much more strategic level, this process completely undermines the integrity of the data. Once data is put into Excel, the business user is violating most data governance policies. The data is no longer trusted.
Harnessing Business Assets and Them Making Actionable
Many organizations have come to rely on dashboards as a way to make data actionable. Yet, even though an organization has decided to show business intelligence data via a graphical dashboard and/or scorecards, how do they include all the information needed into that dashboard? The Computing study demonstrated just how pervasive this problem can be. According to the financial decision makers questioned, 52 percent felt that less than half of the information needed to run the business effectively could be pulled from existing documents and in the correct format without having to rely on IT. The fact that users struggle to get a grip on BI software is perhaps reinforced by the fact that for two out of three of those polled, it is not the more tech savvy members of the IT department who are using it. More than half (54 percent) said responsibility for producing BI reports was with individual business departments such as finance and sales. Worse, only 11 percent of these individual executives or business managers said they run reports themselves. This indicates they had to submit a request to the IT department to perform these tasks on their behalf.
Report Analytics is a smart approach in today’s economy. It is an easy, self-service solution which enables business users and data consumers to get the data they need out of existing reports. Mining this valuable intelligence doesn’t necessarily imply that pushing that data into a dashboard is a complex, time-intensive process. The use of Report Analytics helps users to easily visualize report-based data. Not only does this approach leverage an organization’s significant investments in enterprise applications, but it also avoids the costly route of creating a data warehouse, or worse, programming one’s way to an acceptable solution.
About the Author
Michael Morrison is President and Chief Executive Officer at Datawatch Corporation, a provider of report analytics products and services. For more information contact Michael at Michael_Morrison@Datawatch.com or visit www.datawatch.com.