• Votes for this article no votes for this yet
  • Dashboard Insight Newsletter Sign Up

Unstructured Data 101: Project and Technical Success Factors
The third in a series

by Lyndsay WiseThursday, July 12, 2007

Partnerships between BI vendors and search providers bring the possibility of unstructured data analysis to the forefront of organizations. Until this year, enterprise search use was centered on the Internet and within content management systems. With the expansion of these capabilities to include BI, the importance of analyzing unstructured data to help drive performance is gaining momentum. As more search tools become embedded within organization-wide applications, the possibility of leveraging unstructured data is becoming realistic. This means that within the next two to five years, this market will expand and BI search, text analytics and other practical applications of unstructured data will become common within organizations -- and especially within organizations that deploy business intelligence solutions.

Organizations are starting to understand and see the value of embedding search tools within their BI applications to allow end users access to developed reports and applications. For organizations considering adding this functionality or other text-based analysis to their current infrastructure, the following business/project and technical requirements should be considered:

Business and project-related factors

Business factors represent the common factors organizations should consider when commencing any technology project. Although these are intuitive, many organizations fail to follow these basic steps as a precursor to a project, thereby increasing their risk of failure. The list below identifies general considerations and is by no means comprehensive. However, these factors provide organizations with a general guideline of first steps to take when initiating a project to leverage the organization’s unstructured data. By focusing on considerations from a project standpoint, implementation, buy-in, and momentum are easier to gain and to maintain throughout the full project lifecycle. The initial consideration of business factors also allows organizations to gain a solid foundation, to build a roadmap, and to identify the business issues that are at the forefront of the change initiative.
  1. Business pain identification

  2. The first step in any initiative is to identify the business pain. This means acknowledging the core issue that precipitates the need for change. A simple example for BI search is the fact that many organizations’ use of BI is limited to super users or end users that know exactly what information they require and where to find it. Other employees looking for sales or financial data may not know where the information resides. The use of BI search allows any user to access BI-related data with simple keyword searches to expand the range of use of business intelligence within the organization. The business issue associated with end users being unable to access the information they need boils down to money. The time and money spent on implementing a BI solution that is not accessible makes the advantages of BI outweighed by the lack of ease of use and access to information. Consequently, the value associated with the expansion of the current BI solution to include other departmental requirements or to add performance management functionality may be met with resistance because of the lack of optimization of the current system.
  3. Management buy-in

  4. Attaining management buy-in is an essential aspect when looking to implement any solution or when starting any new project. To attain buy-in, organizations should identify the proper stakeholders at the beginning of the initiative to ensure support throughout the project. Additionally, each stakeholder identified should feel a personal connection to the initiative to attach future solution benefits to their department or future related initiatives. This involves attaching the business issue and future benefits directly to each stakeholder. Two types of departments to target are those tied to business intelligence and those that will benefit most from unstructured data analysis. The use of unstructured data determines stakeholder interest. If an organization chooses to begin with BI search or sales and marketing-based analysis, then the heads of those departments should be involved in the process.
  5. Solution roadmap

  6. Developing a roadmap includes the identification of steps involved to actualize the project and to link individual tasks to project completion. Examples include needs analysis, requirements gathering and milestones to make sure the project stays on track. Although some roadmap items are more technical in nature, they provide the gap between business and IT to allow organizations to identify potential resource conflicts before project kickoff.
  7. Strategic alignment between business and IT

  8. The internal alignment between business units and the IT department is a topic that is discussed at length. Unfortunately, ways to actualize and to capitalize on the relationship between business and IT are dependent upon individual organizations as no one solution exists. To implement any technology solution that is aligned with the business and focused on increasing performance within the organization, there needs to be cohesion between business and IT. Within the realm of unstructured data, this is more so because the ability to capture and to present the appropriate data is IT’s responsibility. However, the identification of what information is needed and what is valuable to the business (such as customer satisfaction, or customers’ comparative views of competitors’ products) should be identified by the business unit, making the cohesion between both units an essential aspect to project success.
  9. Change Management

  10. The focus on continual improvement and helping end users do their job better is part of what makes an organization maintain competitive advantage. With BI search, the buy-in required to initiate change may be easier than through other initiatives. Search tools offer employees easier access to data in a way that is consistent with their personal computer use. This means that users are more likely to adopt this usage of BI over other organizational applications of business intelligence. Also, an organization’s proactive approach to change and a focus on improving processes enables employees to maintain a forward-looking approach, allowing organizations to be more open minded to future initiatives involving unstructured data.

Technical considerations

Once organizations identify and consider the business factors that drive the process of change, organizations should align their roadmap with the required technical considerations. Technical considerations help organizations move beyond simple discovery of what the system requirements are or what the current architecture is, to help identify the potential factors that affect implementation. By reflecting on these factors beforehand, organizations are more prepared to handle issues that arise.
  1. Integration between BI and text analytics or text mining vendors

  2. Many BI vendors have data mining and text mining features built into their applications. Common uses include the detection of patterns and trends and the identification of what information exists and where it resides. When organizations adopt best-of-breed solutions, the ability to integrate text analytics into the existing BI infrastructure should be considered. Many vendors develop their applications to integrate easily with specific vendors based on their client base. Organizations should identify how the best-of-breed solution integrates with the current infrastructure to anticipate the amount of time needed to implement the solution. Additionally, organizations should speak with other customers as references to identify the details of integration between the two platforms.
  3. The data warehouse’s ability to handle of unstructured data

  4. Data warehouses were developed to handle structured data. Within BI, the concept of unstructured data has been relegated to BI search tools to find information from developed reports using structured data. Consequently, as BI environments mature, and organizations move beyond simple data mart builds and sales-based analyses, there is a move towards identifying ways to leverage other useful information that may be important for organizations to solve key business issues. Therefore, the way in which data warehouses are built and used will evolve with the organization to meet these requirements. Currently data warehouses are not equipped to handle unstructured data; however, organizations can look for alternative database solutions and can develop applications that will handle future requirements.
  5. Use of search

  6. The use of search within BI has become more commonplace, especially as vendors partner with Google, Yahoo!, and other search providers to bring unstructured search to the world of BI. As these technologies become more pervasive in the industry, their use will expand within organizations. Just as enterprise search is prevalent within content management systems and has expanded to include business intelligence, the use of BI search will expand within other areas of the organization as well. Because taxonomies may be built into the BI solution structure, extending search capabilities may or may not be intuitive. Organizations should consider the potential value search will provide other areas within the organization as they develop initial solutions for BI as the demand for search may grow.
  7. Transferable applications/solution growth
  8. Successful solutions are generally adopted by additional departments within the organization. The world of scorecards and dashboards provide a good example of technological uses that have expanded within the organization to include sales and marketing, operations, HR, etc. Organizations should consider practical applications of search and unstructured data to identify future use of these applications within the organization.
  9. Volume of data
  10. The volume of unstructured data collected within organizations can increase exponentially over time. Extra storage may be required when implementing a new solution to meet the technical requirements of that solution. Additionally, the sheer volumes of unstructured data mean that to house or to access that data it is important to have the proper infrastructure and processes in place. This includes considering the options to both storing and to extracting the appropriate information for analysis. Search applications that are embedded within current applications do not require the same considerations as a text analytics initiative where the identification of the required data, the capturing of that data and its analysis require extra storage/server space.

The business and technical factors identified provide an overview of what organizations should consider when looking to implement a new solution or to expand their current applications to include the use of unstructured data. Whether through search or other forms of unstructured data analysis, organizations should identify the business requirements, attain the appropriate buy-in and include the stakeholders within the business units affected by the change. Business factors should be considered first, but the importance of technical considerations should not be underestimated, as a balance between the two is what can help ensure project success.

(Copyright 2007 - Dashboard Insight - All rights reserved.)

About the Author

Lyndsay Wise is a senior research analyst for the business intelligence and business performance management space. For more than seven years, she has assisted clients in business systems analysis, software selection and implementation of enterprise applications. She is a monthly columnist for DMReview and writes reviews of leading technologies, products and vendors in business intelligence, data integration, business performance management and customer data integration.

Tweet article    Stumble article    Digg article    Buzz article    Delicious bookmark      Dashboard Insight RSS Feed
Other articles by this author


No comments have been posted yet.

Site Map | Contribute | Privacy Policy | Contact Us | Dashboard Insight © 2018