Ben Shoemate's Notebook

A while ago something broke inside me. My deepest held beliefs were shattered. I started drinking again. Absinthe. I love absinthe. 69% ABV.

It started when I saw it. A glorious pie chart displaying growth rates. Growth! At first, I couldn’t believe it. I took off my glasses. Yes, growth rates. In a pie chart. I had to try it. Unemployment rates. Growth rates above 100%. Apples and oranges. Trends. Pies, pies, pies. And absinthe.

Now I’m playing with doughnuts. Mountains and trenches, from the Wikipedia. I don’t like it. It murmurs. Too much signal. I want pure noise. Maybe a 3D effect can help. I’ll try again.

Bye, Tufte. Bye, Few. Bye, Bertin. I’m almost free.

Oops. April 1 is almost over. Back to business then.

 

______________________

Post from: Excel Charts Blog.

Highest Mountains, Deepest Trenches, and a Doughnut

No related posts.

I imagine public interest in the idea of IBM software winning a quiz show is starting to wane–and remember, I taped these shows a month in advance, so I’m thirty days ahead of the attention-waning curve. Here, while anybody might still care, are six Watson stories I never told.

1. IBM research labs have no dressing rooms for some reason! As a result, Brad Rutter and I took over two HR conference rooms to change clothes, get made up, etc. I’ve already emailed the HR department a little chart showing which parts of their desks I sat on naked. Sorry guys! Hope the basket of muffins made up for it. Also, the whole place was designed by superstar midcentury architect Eero Saarinen…but it’s got like two men’s rooms in it. What, in Finland nobody needs to pee? Because Brad and I were supposed to be strictly sequestered from the Watson team, keeping IBMers out of “our” men’s room became a full-time job for the contestant coordinators. One guy who got yelled for trying to use his own restroom turned out to be senior vice president John E. Kelly III. “I don’t think anyone’s said no to him in years!” said one white-faced IBM employee.

2. Watson is mostly written in Java. After one of the practice games wound up, I sat down in the auditorium behind Watson’s operators, hoping to sneak a peek at what they were up to. The first thing I saw was a whomping Java error trace someone was trying to debug. As a Java programmer for many years, this was both exciting and horrifying: my own tools had turned on me! Luckily, like any good craftsman, I choose not to blame my Yogi Berra.

3. Garry Kasparov didn’t make the cut. In the first game, Watson nailed a clue about Garry Kasparov’s defeat at the “hands” of Deep Blue, eliciting a burst of applause from the deeply-in-the-tank studio audience. Unfortunately, home viewers never got to see this IBM bloodlust in action. The Kasparov clue, like maybe half a dozen others over the course of the taping, had to be tossed out for technical reasons. The Jeopardy! crew’s 26 years of experience doing their show means they normally run a pretty tight ship, but the added complications of (a) doing the show on the road, and (b) connecting to a computer opponent for the first time meant endless glitches. I think Brad and I both wonder if I we could have eased into a better buzzer rhythm if we hadn’t had to stop tape every category or two.

4. Jerome Vered was pissed. I wrote in Slate that I was the first human made obsolete by Watson, but that’s not strictly true. Jerome Vered has that honor. You may recall that the last time Brad and I played Jeopardy! was the finals of the Ultimate Tournament of Champions, in which the third man was L.A. gadabout and quiz show veteran Jerome Vered. One of the Jeopardy! contestant coordinators said that, just hours after the lineup for the Watson match was announced, she got an email from Jerome: “So you replaced me with a computer?!?” For some reason I like to imagine him saying this with gloomy equanimity, like Eeyore.

5. Alex Trebek was pissed. Between the practice rounds and the televised game, Watson switched up its strategy dramatically–most notably, it started hunting for Daily Doubles instead of marching down the categories in order. The reason was simple: Watson comes with a practice mode and a game mode, and it wasn’t playing in game mode yet. I don’t think Brad or I felt like this was unfair–after all, we could have concealed strategy from Watson in the practice rounds as well–but some of the Jeopardy! powers-that-be felt the change-up was a bit of a hustle, since presumably one of the reasons for the practice rounds was to let us see Watson’s gameplay in action. “Alex is pissed,” Stephen Baker told me the weekend after the taping, right after he got off the phone with a still-hot-under-the-collar Trebek.

6. Alex Trebek was messing with my head! Jeopardy! “Clue Crew” stalwart Jimmy McGuire stood in for Alex as host of the practice rounds, so that audiences seeing clips wouldn’t assume they were watching the actual match. Alex, though, was so interested in watching Watson in action that he drifted into the crowd in his shirtsleeves and watched the practice rounds from the front row. This was oddly disconcerting! “Alex, I can’t play with you watching!” I shouted to him. “You’re in the wrong spot.” It was, I thought, exactly like trying to pee at a urinal with someone watching you. Or with a Jeopardy! contestant coordinator kicking you out of the men’s room, I suppose.

(Watson likeness by Matt “Matsby” Page.)

Hans Rosling of GapMinder is one of my heroes. He has become an engaging and powerful teller of quantitative stories. He’s making a difference in the world. Even the most talented among us, however, sometimes slip up. Rosling’s recent video, produced by BBC Four, takes advantage of technology to place him behind a transparent bubble chart, making it possible for him to direct our attention to particular items with greater ease and clarity, without blocking our view-a worthy goal for statistical narrative. This approach suffers, however, from one significant flaw: in addition to Rosling, an entire room with bright lights, beams, and windows appears in the background as well, resulting in a great deal of distraction.

The production crew could have easily used a clean backdrop for the video, which would have removed all distractions and made it easy to focus on the data and Rosling’s narration. This problem perhaps never occurred to the technicians (although it should have), and I suspect that Rosling had no idea that all those windows and lamps with glaring lights would show up in the finished video. Attention to these details, however, makes the difference between fun and engaging data visualizations that tell stories effectively and those that feature novelty and entertainment over substance. To focus attention on the story, all distractions must be removed. As we venture into new opportunities that technology makes possible, we dare not forget the important lessons of the past. In the years since Edward Tufte began promoting the reduction of non-data ink in visual displays, research has confirmed the importance of this practice due to limitations in human perception, cognition, and memory. We can only focus on a small portion of the visual field at one time, we can only consciously attend to one task at a time, and we can only hold about three chunks of visual information at a time. There is no room for distraction of any kind. Anything that isn’t data must have a good reason to be there or it should be eliminated. As Antoine de Saint-Exupery once wrote, “Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.” This is especially true when telling quantitative stories.

Take care,

I delivered a keynote presentation at Tableau’s Customer Conference last week. Several people at the conference expressed appreciation for the insights contained in one of my slides in particular, so I thought I’d share it here in my blog.

Here’s what I said while showing this slide:

The industry that has claimed responsibility for helping organizations get real value from information goes by the name “business intelligence.” This term was originally coined by Hans Peter Luhn, an IBM researcher, way back in 1958. Luhn defined business intelligence as “the ability to apprehend the interrelationships of presented facts in such a way as to guide action towards a desired goal.” The term didn’t catch on, however, until sometime after Howard Dresner, best known for his work at Gartner, used it again to breathe new life into to the data warehousing industry. Dresner defined the term as “concepts and methods to improve business decision making using fact-based support systems.”

Contained in these early definitions was the seed of an inspiring vision that caused people like me to imagine a better world, but the business intelligence industry has done little to help us achieve the vision of the people who coined the term. When Thornton May was interviewing people for his book “The New Know”, he asked a prominent venture capitalist known for his 360-degree view of the technology industry what he thought of when he heard the phrase business intelligence. His response was “big software, little analysis.” Sadly, his response rings true.

In the 1990s, the data warehousing industry, which had become lackluster due to its many failures and the inability of thought leaders and vendors to tell us anything new and worthwhile, promoted the term business intelligence (BI) as its new rallying cry. It was used as a marketing campaign to rekindle interest in old technologies, but did little to change the course of events. The industry continued to focus on building the infrastructure of data rather than the tools and methods that are needed to actually use data. Until this day the BI industry still focuses on collecting, cleaning, transforming, integrating, storing, and reporting data, but the activities that actually make sense of information and use it to support better decisions have remained behind a wall that they’ve failed to scale and have never seriously tried to scale. For information to be useful, we must explore it, analyze it, communicate it, monitor it, and use it to predict the future, but the BI industry’s attempts to support these activities with few exceptions have been tragically comical. The technology-centric, engineering-oriented perspective and skill set that has allowed the industry to build an information infrastructure is not what’s needed to support data sensemaking. To use the data that we’ve amassed, a human-centric, design-oriented perspective and skill set is needed.

All of the traditional BI software vendors and most of the industry’s thought leaders are stuck on the left side of the wall. The software vendors that are providing effective data sensemaking solutions—those that make it possible to work in the realm of analytics on the right side of the wall—have come from outside the traditional BI marketplace. Vendors like Tableau, TIBCO Spotfire, Panopticon, Advisor Solutions, and SAS tend to either be spin-offs of university research or companies that have ventured into the BI marketplace from a long history of work in statistics. Traditional BI software vendors and the scores of recent start-ups that emulate them can choose to climb the wall, but it won’t be easy. They’ll need to rebuild their approach from the ground up. Unfortunately, most of them don’t even realize that their attempts to provide data sensemaking solutions are embarrassingly uninformed and ineffective. Until they see the wall, they’ll never learn to scale it.

Take care,