Business intelligence analyst Wayne Kernochan, of Infostructure Associates, says it's a shame exploratory data analysis (EDA) doesn’t get more attention from enterprises looking to increase their competitive edge.
Business intelligence has taken a step forward in maturity over the last few years, as statistical packages have become more associated with analytics. SAS has for years distinguished itself by its statistics-focused business intelligence solution; but when IBM acquired SPSS, the grand-daddy of statistical packages, the importance of more rigorous analysis of company and customer data seemed both confirmed and more obvious.
Moreover, over the years, data miners have begun to draw on the insights of university researchers about things like "data mining bias" and Bayesian statistics – and the most in-depth, competitive-advantage-determining analyses have benefited as a result.
So it would seem that data miners, business analysts and IT are on a nice query-technology glide path. Statistics completes the flexibility of analytics by covering one extreme of certainty and analytical complexity, while traditional analytics tools cover the rest of the spectrum up from situations where shallow and imprecise analysis is appropriate. And statistical techniques filter down by technology evolution to the “unwashed masses” of end users.
And yet there is a glaring gap in this picture – or at least a gap that should be glaring. This gap might be summed up as Alice in Wonderland’s "verdict first, then the trial." Both the business and the researcher start with their own narrow picture of what the customer or research subject should look like, and the analytics and statistics that accompany such hypotheses are designed to narrow in on a solution rather than expand due to unexpected data. Thus, the business/researcher is likely to miss key customer insights, psychological and otherwise.
Pile on top of this the "not invented here" syndrome characteristic of most enterprises, and the "confirmation bias" that recent research has shown to be prevalent among individuals and organizations, and you have a real analytical problem on your hands.
For the rest of the article click here.
Credit: Enterprise Apps Today