BI Has Hit the Wall

09 Sep

I delivered a keynote presentation at Tableau’s Customer Conference last week. Several people at the conference expressed appreciation for the insights contained in one of my slides in particular, so I thought I’d share it here in my blog.

Here’s what I said while showing this slide:

The industry that has claimed responsibility for helping organizations get real value from information goes by the name “business intelligence.” This term was originally coined by Hans Peter Luhn, an IBM researcher, way back in 1958. Luhn defined business intelligence as “the ability to apprehend the interrelationships of presented facts in such a way as to guide action towards a desired goal.” The term didn’t catch on, however, until sometime after Howard Dresner, best known for his work at Gartner, used it again to breathe new life into to the data warehousing industry. Dresner defined the term as “concepts and methods to improve business decision making using fact-based support systems.”

Contained in these early definitions was the seed of an inspiring vision that caused people like me to imagine a better world, but the business intelligence industry has done little to help us achieve the vision of the people who coined the term. When Thornton May was interviewing people for his book “The New Know”, he asked a prominent venture capitalist known for his 360-degree view of the technology industry what he thought of when he heard the phrase business intelligence. His response was “big software, little analysis.” Sadly, his response rings true.

In the 1990s, the data warehousing industry, which had become lackluster due to its many failures and the inability of thought leaders and vendors to tell us anything new and worthwhile, promoted the term business intelligence (BI) as its new rallying cry. It was used as a marketing campaign to rekindle interest in old technologies, but did little to change the course of events. The industry continued to focus on building the infrastructure of data rather than the tools and methods that are needed to actually use data. Until this day the BI industry still focuses on collecting, cleaning, transforming, integrating, storing, and reporting data, but the activities that actually make sense of information and use it to support better decisions have remained behind a wall that they’ve failed to scale and have never seriously tried to scale. For information to be useful, we must explore it, analyze it, communicate it, monitor it, and use it to predict the future, but the BI industry’s attempts to support these activities with few exceptions have been tragically comical. The technology-centric, engineering-oriented perspective and skill set that has allowed the industry to build an information infrastructure is not what’s needed to support data sensemaking. To use the data that we’ve amassed, a human-centric, design-oriented perspective and skill set is needed.

All of the traditional BI software vendors and most of the industry’s thought leaders are stuck on the left side of the wall. The software vendors that are providing effective data sensemaking solutions—those that make it possible to work in the realm of analytics on the right side of the wall—have come from outside the traditional BI marketplace. Vendors like Tableau, TIBCO Spotfire, Panopticon, Advisor Solutions, and SAS tend to either be spin-offs of university research or companies that have ventured into the BI marketplace from a long history of work in statistics. Traditional BI software vendors and the scores of recent start-ups that emulate them can choose to climb the wall, but it won’t be easy. They’ll need to rebuild their approach from the ground up. Unfortunately, most of them don’t even realize that their attempts to provide data sensemaking solutions are embarrassingly uninformed and ineffective. Until they see the wall, they’ll never learn to scale it.

Take care,