Organizations have a personality, and it bleeds into everything from executive reporting to product offerings. A recent Fortune article entitled Microsoft without Gates offers this wonderful tidbit about Steve Ballmer, CEO of Microsoft:

Even though he never was a serious computer programmer, by all accounts Ballmer is just as good at math as Gates is. He lives and breathes data. “Steve has a computer in his head,” says Bob Muglia, a 20-year company man who heads the Server and Tools division. Ballmer expects his subordinates to be adept in math as well. He distributes 11-by-17 sheets filled with numbers detailing the progress of various operations. The numerals are so small that executives use transparent magnifier rulers to see them. But there are never any columns showing percentage changes. Ballmer believes people ought to do that in their heads. It saves space on the paper for more numbers.

Wow. If it is as bad as the author describes, Ballmer has designed the anti-dashboard.

The Presentation Zen blog offers another great example of organization culture as displayed in business artifacts.

Gates here explaining the Live strategy. A lot of images and a lot of text...Good graphic design guides the viewer and has a clear hierarchy or order so that she knows where to look first, second, and so on. What is the communication priority of this visual? It must be the circle of clip art, but that does not help me much.

Does it get more "Zen" than this? "Visual-Zen Master," Steve Jobs, allows the screen to fade completely empty at appropriate, short moments while he tells his story.

There is a tendency with reporting, and dashboards in particular, to cram as much information on the page as possible. It is a problem that Avinash describes with typical candor:

“This one of the core reasons why most dashboards are 'crappy', i.e. they are data pukes that provide little in terms of context and even less in terms of actionable value.”

In the past, we have offered tools to make data presentation as clear as possible (chart chooser, Excel chart cleaner). Sometimes clean isn’t enough; a more dramatic approach is needed.

One alternative is to shift the focus from the full data to changes in the most critical data points. By pulling out the important exceptions, you can make it easier for your audience to digest what matters and take action.

Stephen Few says in his book Information Dashboard Design:

“The best way to condense a broad spectrum of information to fit onto a dashboard is in the form of summaries and exceptions…given the purpose of a dashboard to help people monitor what’s going on, much of the information it presents is necessary only when something unusual is happening; something that falls outside the realm of normality, into the realm of problems and opportunities. Why make someone wade through hundreds of values when only one or two require attention? We call these critical values exceptions.”

Alerts are one mechanism to turn the focus to the exceptions, outliers and data highlights. Whether embedded in the dashboard or presented separately, alerts can be the extra layer of abstraction that make a dashboard useful. Unfortunately, they are hard to get right. I’ve arrived at four C’s for effective alerts—context, cogency, communication, control. Here’s a checklist to consider as you build alerts into a dashboard or report:

Context: Users need to understand how an alert is defined and how it fits into the larger picture.

• Are the parameters well defined? An alert is commonly defined by the following factors: metric (e.g. revenue), dimension (e.g. time), delta (e.g month over month change), scope (e.g. Northeast region, Peanut-product line), threshold (e.g. increase or decrease of 10%).
• Is the timing of the alerts actionable? One client explained to us that fluctuations in many of their metrics make monthly alerts too frequent—it would unnecessarily alarm people when, from their perspective, no significant trend had been established.
• Is the change statistically significant? This is of particular importance when you are measuring deltas. A doubling of traffic from a referring site doesn’t mean much when it is moving from one to two visitors.

Cogency: An alerting system needs to avoid causing unnecessary alarm while delivering easy-to-understand information that can be acted upon.

• Can the alerts be described in simple terms that even an executive can understand? Alerts should have a real-world meaning that users are familiar with. If an alert is based on a complex metric, for example, users will be confused as to the implications.
• Is the alert actionable? In the best cases, alerts should point users to both the drivers of the alert and the actions that can address the situation. This system does neither: ![terror warning system]
• Are the alerts so granular and/or frequently triggered that users will get alert fatigue? Excessive use of alerts will undermining their credibility. We saw this happen at one client where an IT-designed system threw off alerts like they were going out of style. The application went out of style the next year when users decided it was more distracting than useful. Here’s another example of a system that seems designed to raise blood pressure.

(It appears that a 5% increase in brand attribute performance isn’t good enough to get you out of the yellow.)

Communication: Alerts must be designed to effectively capture attention and inform.

• Is the alert placed in context? Google Finance does a nice job of putting news alerts within the stock chart.
• Is it clear what the user should do next? Give the user a clear path to more information so they can understand the full context of the alert.
• Does the sophistication of your alerts match the sophistication of your audience? I’ve found that it is better to start with some simple alerts so your audience can begin to learn what they mean and how to react. Over time, these alerts can become more refined and focused to capture complex situations.
• Does the alert draw the eye without being visually overwhelming or annoying? Here’s a article about how to “reduce visual noise” in dashboards.
• Is color used appropriately? Red means bad. Yellow is sorta bad. Green means good (but “good” things don’t need to be alerts). It isn’t particularly fair for color blind folks, but these conventions are deeply rooted.
• Have you found the best mechanism for presenting alerts? Alerts can be sent through e-mail, as SMS message, blasted over the office intercom system, or posted to the wall in the bathroom. What is the most convenient and appropriate medium?

Control: Advanced alert system should give users the ability to customize and manage alerts.

• Can the user identify the important alerts for them, and avoid the others? As hard as you may try in designing the dashboard or report, you aren’t in the shoes of the users. They will learn what they want to pay attention to and what information is extraneous.
• Can the user adjust the parameters? With more sophisticated dashboards, you want to give users the ability to adjust parameters to hone in on the exceptions that really require action.
• Can the user analyze alert frequency and trends? I’ve never seen a system that does this, but having the ability to view and analyze alert history seems critically important to getting a holistic view of performance.

What do you do when you’d rather be out driving your BMW rather than sitting in your corner office? Make a business dashboard that looks like your car dashboard, of course. You’ll want to have lots of tachometers, temperature gauges, and traffic lights. It’s the ultimate business-driving machine.

It isn’t controversial to complain about the ineffectiveness of “gauges” for data visualization. In fact, even some of the worst offenders admit that gauges aren’t ideal:

Dr. Robert Alison of SAS in showing off a new easy graph procedure for creating gauges says:

“I know, I know … gauges have lots of drawbacks in dashboards. But hey, the other philosophy is 'give the customer what they want' … and try to make it work as well as possible. So, as far as gauges go, these are pretty decent.”

Here’s the example he uses to show off “one of the sharper-looking dashboards I’ve seen”

The folks at Business Object’s Xcelcius admit that gauges shouldn’t always be used in their article entitled “The Use (and Misuse) of Gauges”.

That doesn’t stop them applying a triple-coat of carnauba wax while neglecting their rule to always label the endpoints.

In the end, they primly note: “Despite some recent bad press, a gauge isn’t inherently a poor graphic.” Bad press, is it. If only gauges had better PR.

In my opinion, warning about potential misuse isn’t firm enough. Gauges shouldn’t be used except under the most severe threats from a client offering enough money to buy absolution.

Stephen Few, a man who doesn’t mince words on information visualization, says:

“If you squint really hard, you can barely make out some of the values. But who cares, because if you’re an executive who likes to pretend that you’re driving a car while sitting at your desk rather than actually managing your business, then having a dashboard that is truly informative doesn’t really matter.”

Charley Kyd says:

“Using dashboard gauges for management reporting typically is a mistake. Gauges hide information that managers need and consume significant space in a report.”

Let’s break down the problems with gauges:

Gauges hide trends. For all the focus on how a value is performing, you’d think people would care about the historical trend.

Circles aren’t good for showing differences. Like pie charts, circular gauges aren’t the best way to show size or changes in values—bars are a more straightforward, if less sporty, approach.

Space eaters. Often gauges are used to show a single value. All that decoration for a single value must send Tufte into a tizzy. Attempts to cram two values into a gauge can be confusing. How do you read this one?

Difficult to read. The values can be obscured by all the attractive accoutrement:

Ranges can be tricky. By the analogy to a car dashboard, gauges are expected to have a static minimum and maximum value. What happens when a value goes beyond the pre-set range. Here’s an example of the “right way” from Xcelsius with the label: “This gauge shows a retail store’s progress against a daily revenue target.” We can only presume the maximum value is $45,000. What happens if I go beyond $45,000?

Traffic lights are contradictory. I may be getting nitpicky, but I can’t both have my traffic light look like the real thing (red on top, green on bottom) and abide by basic data visualization assumptions (better is higher).

Lastly, there are so many better options. Here’s a beautiful data display (courtesy of Mr. Few) that could have been done with gauges, but mercifully was not.

Misaligned goals, distorted behaviors, and a misguided sense of success... no, I'm not referring to college graduates. I'm talking about the problems caused by using the wrong metrics in your organization. You've probably seen examples like tracking average customer profitability and losing perspective on the variance in profitability or evalutating customer service reps on calls handled without regard for the quality of the experience. I'd like to offer up a quick-bake recipe for choosing the right metric.

Step 1: Set the context

Metrics generally serve one of two purposes. Start by understanding what you are trying to achieve.

1. Identifying problems. Defining the right metrics in this case requires you to do a little detective work: What is the data residue of a problem? What evidence can be found and how exactly does it show up?

2. Measuring performance. The right success metrics need to focus on measures that can be controlled and where improvement in the number is unabiguously a good thing.

Step 2: Balance the four dimensions of a good metric

Lots of metrics fail in at least one of these dimensions. A few examples:

• Common interpretation: We had a client who made a distinction between "leads" and "prospects" in their marketing organization. Prospects had theoretically expressed more interest in the service through their actions. Unfortunately the line between leads and prospects was always hard to decipher and the definitions were hard to communicate. On a related note, we got a kick out of Tom Davenport's (author of "Competing on Analytics") assertion that a company competing on analytics needs to "invent proprietary metrics for use in key business processes." There is nothing inherently wrong with "invented proprietary metrics" but it sounds like something that is designed to confuse anyone outside of the inner sanctum.
• Actionable: Metrics are frequently too broad for the impact that a particular group can have. Customer satisfaction is a popular dashboard staple, but it is hard for most managers to see how they can have a significant impact on the number.
• Accessible, credible data: Sometimes the most valuable and obvious metrics are frustratingly hard to track. In the web analytics world, unique visitors is important to know, but user deletion of cookies has thrown a wrench into the works.
• Transparent, simple calculation: Top NFL agent Leigh Steinberg says of the famous quarterback ratings metric:"Other than one attorney in our office, I am unaware of a single human being who has the capacity to figure a quarterback rating." I don't know what kind of art majors he hires, but all they need to do is use the simplified formula: (83.33 * Comp %) + (4.16667 * Yds per att) + (333.333 * TD pct) - (416.667 * INT pct) + 25/12.

(Want a little validation of this framework? Avinash, respected web analytics guru, just published a post with "Four Attributes of Great Metrics" and he landed on a strikingly similar set of four: 1) instantly useful (i.e. actionable); 2) relevant (i.e. common interpretation); 3) timely (i.e. accessible); 4) uncomplex (i.e. transparent and simple).)

Step 3: Avoid the metrics bugaboos

Finally, here are a few traps that I've seen in deciding on appropriate metrics:

• Trending and distributions: Don't always try to compress a metric into a single number. Often it is more revealing to show the metric across time or as a distribution to uncover variance.
• Edge cases: There will always edge cases where a metric may not mean what you think it means. These situations are worth understanding, but you shouldn't allow the perfect to be the enemy of the good.
• Setting goals: Could you hold someone accountable for this metric without them throwing out a half-dozen reasons why it doesn't make sense? It's a decent test of the value of the metric.
• Self-serving: Be careful that you don't select metrics simply because you know they'll make you look good.

Charts are a great way to explore data. Here is some American baseball data showing player salaries over a five year period.

Charting this data with a line chart would allow us to see trends in salaries by team. However, when we use Excel’s default chart, we get something that looks like this:

That’s quite a mess. It would be a lot easier if we could create one chart for each row.

The OFFSET function is going to help. In its simplest form the OFFSET function works like this:

OFFSET(anchor, rows from anchor, columns from anchor)

That is, OFFSET will start with the anchor cell, go down a number of rows from that anchor and over a number of columns and return the value it finds.

We can use the OFFSET function to create cells that pull a single row of data out of the table dynamically. We create a new row atop of our data and create a series of OFFSET functions that all rely on a single cell (the big yellow one) for their row offset. So changing one cell will pull different rows of data into our fixed location.

Now, chart the data that doesn’t move.

After fixing the chart, we’d like to make it easy to change the value in the big yellow cell.

We can use Excel Forms to build a lightweight user interface. Bring up the Excel forms toolbar by rightclicking on any toolbar and choosing Forms. Place a scrollbar beside the chart.

Right clicking on the scrollbar allows you to Format Control. Link the control to the cell that is controlling all the row offsets. Now, moving the scrollbar will update the chart.

Now, the scrollbar controls the chart. Here is the baseball spreadsheet for you to play with: Baseball_offset.xls Have fun!

Note: this post is adapted from a presentation I gave at eMetrics 2007 in San Francisco.