• Votes for this article no votes for this yet
  • Dashboard Insight Newsletter Sign Up

Big data - what it means for data visualization and dashboard applications

by Lyndsay Wise, President, WiseAnalyticsThursday, September 1, 2011


Dashboards are one of the most popular BI tools, representing a front-end access point to more meaningful analysis and business visibility. But they are just one aspect and many organizations overlook the data requirements needed for any successful BI solution. Requirements include consolidating multiple data sources, maintaining a centralized data warehouse, and managing data quality over time. These contribute to creating an environment that can support end-user service level agreements (SLAs). This is a complicated and time-consuming process, but the reward is great in the end.

Organizations that maintain a strong data infrastructure can plan more effectively, track their products through the supply chain, and manage relationships with their sales channels to proactively identify opportunities or potential problems. Dashboards and the application of data visualization techniques provide easy access to business information, which is derived from large volumes of data. But BI tools including dashboards without this data layer have limited value to companies.

With organizations storing increasing amounts of complex data, the role of data warehousing and general data management is becoming more intricate in its design and maintenance.   Intended to address these challenges, concepts like big data, data virtualization, and data warehouse appliances are becoming the now common.

Companies store multiple terabytes and petabytes of data and need timely access to advanced analytics derived from this information. Playdom, an online social gaming company owned by the Walt Disney Company, uses Netezza, a data warehouse appliance, to collect two terabytes a day from eight million daily players. Playdom uses Tableau Software, a visual analytics solution, to provide Playdom’s 80 licensed users with instant access to data.  Without that software, they wouldn’t be able to gain insight from their large collection of data. This highlights the fact that data is only as valuable as how it is accessed.

A look at big data

There are different definitions of big data depending on its application and the vendors that provide support for its use. Research group Gartner Inc.’s definition of big data includes a vendor’s ability to provide support for data volumes, variety, and velocity to facilitate an organization’s effective management of large data sets that require complex processing.

The actual achievement of this goal  is elusive to many. Technologies like Hadoop and MapReduce provide an access point to distributed data and large data sets, but this remains a new science. Vendors and organizations alike still grapple with how to best use it as simply storing and accessing diverse and complex data sets is not enough.  These data issues and challenges to the ability to manage information assets help explain the slow adoption of big data.

Big data’s push into business intelligence and the importance of self-service dashboard use

In the past, BI made big promises including quick access to large data sets, centralized data warehouses, and advanced analytics. In reality, however, traditional BI offered limited benefits. Organizations were unable to access large data sets or even load large amounts of information without applying batch processes that could take many hours or even several days. Once the data was loaded, it was often stale, and interactive analytics were limited to a specific number of rows and columns and only available to those users with high-end computer skills and analytical capabilities. The gap between what BI promised and what companies actually got was so large that the expected ROI was never realized. But companies can finally get insight out of their information with big data.

Over the past couple of years, data warehousing technology, data integration techniques, and analytics capabilities have advanced substantially. With columnar databases, in-memory analytics, and easy-to-use self-service data visualization solutions, organizations can store and analyze more data in less time.  In addition, the lower cost of storage, software, licensing, and support, has made BI accessible to more organizations. Organizations with large data sets and complex analytics requirements can now easily create something valuable for their organization that meets their ROI requirements.

With all of the talk surrounding big data, its relevance to dashboards might seem obvious. Dashboards make the data more accessible to a larger audience including tech savvy users, executives, and general consumers. Dashboards are also central to the deployment of large sets of information to a variety of users over time.

But this view of big data and its applications are not enough for Chris Twogood, Director of Product Marketing at Teradata, a leading provider of data management technologies. Twogood says his company’s recent acquisition of Aster Data, a provider of analytics and data management for unstructured data, aims to give Teradata customers the benefits of big data envisioned under the Gartner definition explained above.

Add to the company’s commitment to storing and managing large datasets and Teradata seems intent on giving customers the ability to develop a full data management infrastructure to support BI use and information analysis.

“MapReduce helps provide new ways to analyze data and see benefits from it,” Twogood says, with “new types of analytics; patterns, path analysis, etc.”

Teradata and others still face the challenge of developing technology to enable and support advanced analytics. This will require strong data infrastructures and advanced data warehousing on the backend. On the front end, advanced analytics will require strong data visualization tools to interact with the information.

Big data’s relevance in today’s market

The growing importance of analyzing big data, not just storing it, is a reality for organizations..  This is why it is important not to overlook the importance of data visualization.

Elissa Fink, vice-president of marketing at Tableau Software puts it best:  “As people want to slice business and customer behavior and performance in finer detail they can no longer use a subset of data to do so.”

As she points out, organizations need to look at all of their data to get a broader view of how information relates to performance. With the help of visual analysis tools like dashboards, big data access is slowly giving organizations the ability to do that.

General industry trends and implications

The following trends are emerging to support the increased adoption of big data as well as dashboards:

  1. Data warehouse vendors and dashboard providers are forming partnerships to provide full-scale solutions for organizations. As more partnerships develop within the next couple of years, actual big data solutions will emerge to enable organizations to access complex analytics through dashboards.

  2. Real-time/right-time data insights that take into account increasing complexity and different types of data are closer to becoming a reality. The mix of operational dashboards and data warehouses that support continual data updates means that companies will be able to integrate the two to get access to streamed datasets and create forward-looking analyses based on this information.

  3. Self-service dashboard solutions that support large data sets will help organizations interact with more information and, at the same time, provide insights to a wider range of users.

All of these trends relate to providing better insight based on storing larger data sets and accessing more complex analytics. BI technologies are slowly moving towards enabling businesses to get the information they need regardless of data type or size. As data warehousing technologies and data visualization solutions continue to advance and provide better access points to analytics, the use of big data and dashboards to create broader analytics will increase the overall value of BI to the organization.

Tweet article    Stumble article    Digg article    Buzz article    Delicious bookmark      Dashboard Insight RSS Feed
Other articles by this author


No comments have been posted yet.

Site Map | Contribute | Privacy Policy | Contact Us | Dashboard Insight © 2018