• Votes for this article no votes for this yet
  • Dashboard Insight Newsletter Sign Up

Unstructured Data Revisited

by Steve BogdonThursday, February 3, 2011

During February Dashboard Insight will be taking another look at "unstructured data", which was a surprisingly popular topic when we looked at it in September of 2009.  We first invited authors to contribute articles about unstructured data only a few weeks ago, and within days we received a flood of content rich material that will keep our audience busy reading all month long.

"Search" seems to be the most commonly used application in terms of leveraging unstructured data within BI, but there are also ways of structuring unstructured data within data warehouses - plus there is the issue of how all of this fits into analytics. Some hot topics currently surrounding unstructured data include social networking analysis (web analytics) and sentiment analysis – basically taking web content or call center logs and developing metrics surrounding the data collected. 

With blogs, email, SMS, etc., how much of this "noisy text" is actually useful?  What is being done to expand BI capabilities in this area?  Where is it all going?  Learn how you can make use of unstructured data to make better business decisions as we explore this topic in greater detail all month long.

As always we welcome your contributions, and we would like to remind you to discuss these articles under the comments section (after every article).  We also invite you to submit content (an article, white paper or perhaps take part in an interview) to help share your knowledge on how to leverage unstructured data.

Please find this months articles listed below.

Unstructured Data 101

Due to the breadth of unstructured data solutions, whether text or search based, organizations are hard pressed to identify which solutions can offer them the right functionality to help them address their business information pains. However, with a general understanding of the options available, organizations can develop solutions tied to ROI and can increase their strategic initiatives within the organization.

by: Lyndsay Wise, Wise Analytics

What To Do With Your Ugly Measures!

In a nutshell, measures are ugly when they fail to inform your decisions about whether or not you’re getting the results you wanted, and how well your actions are doing in making those results happen. Measures are ugly when they fail to give you the feedback you need to have more control or influence over the results you most passionately want, or need, to create.

by: Stacey Barr, Performance Management Specialist

The Emergence of Hadoop

In order to deal with their increasing load of unstructured information, many business enterprises with industrial strength BI requirements are betting on a project administered by the Apache Software Foundation named Hadoop. Hadoop is an open source software framework that supports the lightening fast processing of huge amounts of data (especially unstructured data) via massively parallel processing (MPP) on distributed clusters of servers: more specifically, “shared nothing” commodity servers which can host and process terabytes (dare I say petabytes) of data over hundreds or thousands of managed nodes.

by: William Laurent, William Laurent, Inc.

Business Intelligence And Unstructured Data

As a rule, analysis software requires much higher data quality than operational software. A typing error in a customer name in an invoice is not enough to break the transaction in most systems. But when it comes to analyzing the data, much more precision is required. In fact, the issue of data quality was identified by The BI Survey 9 as the most common problem in business intelligence projects.

by: Barney Finucane and Michael Schiklang, BARC

Data Analysis and Unstructured Data

Hype surrounding business intelligence and unstructured data ebbs and flows. A few years ago analyzing customer sentiment through text-based analytics was used by a subset of organizations, and independent vendors offering text analytics solutions represented a niche industry that was external to mainstream BI. Organizations adopting text analytics tend to apply solutions externally to BI or structured data analytics. Generally speaking, their goals are to identify customer sentiments, potential risk, and overall trends identification. In some cases companies do integrate various types of analytics to get a broader picture of what is happening within the organization.

by: Lyndsay Wise, Wise Analytics

Unstructured Data Need Not Impede Business Modeling

Most data sources are not within easy reach of IT. Generally, the most valuable data needed to build a driver-based model of a company originates in Operations. Here one can expect to find spreadsheets, as generations of Operations managers have used spreadsheets as analysis tools. Additionally, as a result of the ubiquity of Microsoft Excel, these spreadsheets have been revised and disbursed as regularly as clockwork. Despite this ‘lack of control,’ this unstructured information is likely the most critical data within the company.

by: F. Shan McAdoo, Riverlogic

Unleashing Unstructured Data’s Value

Unstructured data is everywhere. During 2010 the internet saw 107 trillion emails, 152 million blogs, and 25 billion tweets. The lion’s share of our verbal communications and broadcast media has been moving to our websites, intranets, social media and emails. As a result, the productive economy is moving faster than our analytical models can keep up. Being proactive with our messaging and getting the right information into the right decision-makers and buyers isn’t just a matter of buying tools.

by: Kate Pugh, Newvantage

The New 80-20 Rule, Getting Value Out of Your Unstructured Content

Everyone knows the 80-20 rule – that fantastic rule of thumb that guides many of us through our daily decision-making lives. One way to interpret the rule is that a relative few things (the 20%) are important and that if we focus on understanding or managing those things, we’ll get 80% of value. When it comes to information management, common wisdom suggests that the 20% of enterprise data that is structured represents 80% of the value. Yet with explosion of information from new sources such as the web - structured data alone is not sufficient. With new techniques for managing unstructured (such as text, Word, PDF, XML, image, and audio files) and semi-structured content (unstructured content with some meta-data describing the document’s content), it’s becoming clear that the value of this information can be accessed by all organizations.

by: David Caruso, Endeca Technologies Inc.

Can You Trust Your Data?

Managers know that to successfully run a business, you must also take into account a large amount of “unstructured” information. This information includes customer relationship management (CRM) and call center notes, analyst reports and emails. This led to the rise of text mining products to be able to take this “unstructured” information and classify it for use in downstream data analysis.

by: Catherine van Zuylen, Attensity Group

Marketing Analytics and the Use of Unstructured Data to Improve Performance

Sales and marketing efforts are the backbone of an organization. To drive sales, marketing initiatives are executed, and to plan marketing campaigns, sales are analyzed. The connection of both requires the proper management of information, the analysis of that information and the execution of marketing campaigns based on the results. Although organizations spend a lot of time and money within these areas, as the market evolves to include advanced analytics, organizations have to move beyond traditional forms of marketing analytics toward incorporating unstructured data analysis as well. But how do organizations do this and what are the real benefits?

by: Lyndsay Wise, Wise Analytics

Unstructured Data 101: Practical Applications

The analysis of unstructured data is becoming more prevalent as organizations move beyond traditional forms of analyses to gain competitive advantage within the marketplace. Capturing transactional data and sales figures is no longer enough to keep organizations competitive or to maintain customer loyalty and satisfaction. This is especially true as unstructured data represents over half of an organization’s information sources. To stay competitive, organizations are shifting towards the analysis of unstructured data to gain additional insight into their customers, suppliers and competitors. The question organizations should ask when considering the analysis of unstructured data is how does leveraging unstructured data translate into better competitive advantage, increased customer loyalty and improved performance?

by: Lyndsay Wise, Wise Analytics

The Road to BI and Text Convergence

Organizations with mature BI infrastructures constantly look for ways to increase the efficiency of their current applications and to integrate business intelligence into their overall business processes. One way this can be achieved is through the integration of BI with other applications such as accounting, HR, ERP, and CRM. As BI becomes integrated within these applications, the potential for additional data capture becomes apparent. With ERP related processes and manufacturing, organizations can identify parts defects and the reasons why. Within CRM, customer service notes can be captured to help measure customer sentiments. Both of these examples highlight the use of unstructured data to enhance the ability of business intelligence to help organizations with their analysis of their business, including how customers, competitors, and suppliers view their organization.

by: Lyndsay Wise, Wise Analytics

Taking Control Of Your Enterprise’s Unstructured Data

In every enterprise, a wealth of invaluable information exists in unstructured textual form, but organizations have found it difficult or impossible to access and utilize most of it. Although the number varies greatly by industry, it is generally accepted that about one third of vital corporate data is unstructured. (In some of the more service-related industries it is recognized that unstructured information makes up 80 percent of the enterprise’s data.) Business intelligence experts are in uniform agreement that without the contextual information that is provided by unstructured and semi-structured data, a complete view of corporate performance will not be possible.

by: William Laurent, William Laurent, Inc.

The Convergence Of Text Analytics And Business Intelligence

Technology advancements allow business intelligence to be applied more broadly and to include types of analysis never thought possible when it was first developed. This is especially true of unstructured data. In the past, structured data sources were valuable enough on their own to provide organizations what they required to understand their company’s performance and to plan for the future. Now, using structured data alone might not be enough to provide decision makers with a complete view of their company’s landscape.

by: Lyndsay Wise, Wise Analytics

Using Smart Search To Achieve True Business Intelligence

When organizations look at search and business intelligence, many see the use of search only within the parameters of their business intelligence solutions. In this sense, search functionality acts as a powerful tool that lets end users access information that they might not otherwise know exists. For instance, because many people create several reports, end users do not know if information or prepared analyses they require have already been created. Therefore, end users use search to identify what is already available, lessening the work required and adding to the amount of information available.

by: Lyndsay Wise, Wise Analytics

Dashboard Insight will be adding "Unstructured Data" articles all month.

Please check back often.

Data Analysis

Tweet article    Stumble article    Digg article    Buzz article    Delicious bookmark      Dashboard Insight RSS Feed
Other articles by this author


No comments have been posted yet.

Site Map | Contribute | Privacy Policy | Contact Us | Dashboard Insight © 2018