A Dashboard Alerts Checklist

Alert

There is a tendency with reporting, and dashboards in particular, to cram as much information on the page as possible. It is a problem that Avinash describes with typical candor:

“This one of the core reasons why most dashboards are 'crappy', i.e. they are data pukes that provide little in terms of context and even less in terms of actionable value.”

In the past, we have offered tools to make data presentation as clear as possible (chart chooser, Excel chart cleaner). Sometimes clean isn’t enough; a more dramatic approach is needed.

One alternative is to shift the focus from the full data to changes in the most critical data points. By pulling out the important exceptions, you can make it easier for your audience to digest what matters and take action.

Stephen Few says in his book Information Dashboard Design:

“The best way to condense a broad spectrum of information to fit onto a dashboard is in the form of summaries and exceptions…given the purpose of a dashboard to help people monitor what’s going on, much of the information it presents is necessary only when something unusual is happening; something that falls outside the realm of normality, into the realm of problems and opportunities. Why make someone wade through hundreds of values when only one or two require attention? We call these critical values exceptions.”

Alerts are one mechanism to turn the focus to the exceptions, outliers and data highlights. Whether embedded in the dashboard or presented separately, alerts can be the extra layer of abstraction that make a dashboard useful. Unfortunately, they are hard to get right. I’ve arrived at four C’s for effective alerts—context, cogency, communication, control. Here’s a checklist to consider as you build alerts into a dashboard or report:


Context: Users need to understand how an alert is defined and how it fits into the larger picture.

  • Are the parameters well defined? An alert is commonly defined by the following factors: metric (e.g. revenue), dimension (e.g. time), delta (e.g month over month change), scope (e.g. Northeast region, Peanut-product line), threshold (e.g. increase or decrease of 10%).
  • Is the timing of the alerts actionable? One client explained to us that fluctuations in many of their metrics make monthly alerts too frequent—it would unnecessarily alarm people when, from their perspective, no significant trend had been established.
  • Is the change statistically significant? This is of particular importance when you are measuring deltas. A doubling of traffic from a referring site doesn’t mean much when it is moving from one to two visitors.

Cogency: An alerting system needs to avoid causing unnecessary alarm while delivering easy-to-understand information that can be acted upon.

  • Can the alerts be described in simple terms that even an executive can understand? Alerts should have a real-world meaning that users are familiar with. If an alert is based on a complex metric, for example, users will be confused as to the implications.
  • Is the alert actionable? In the best cases, alerts should point users to both the drivers of the alert and the actions that can address the situation. This system does neither: ![terror warning system]
  • Are the alerts so granular and/or frequently triggered that users will get alert fatigue? Excessive use of alerts will undermining their credibility. We saw this happen at one client where an IT-designed system threw off alerts like they were going out of style. The application went out of style the next year when users decided it was more distracting than useful. Here’s another example of a system that seems designed to raise blood pressure.

Lit up dashboard (It appears that a 5% increase in brand attribute performance isn’t good enough to get you out of the yellow.)


Communication: Alerts must be designed to effectively capture attention and inform.

  • Is the alert placed in context? Google Finance does a nice job of putting news alerts within the stock chart. Google Finance
  • Is it clear what the user should do next? Give the user a clear path to more information so they can understand the full context of the alert.
  • Does the sophistication of your alerts match the sophistication of your audience? I’ve found that it is better to start with some simple alerts so your audience can begin to learn what they mean and how to react. Over time, these alerts can become more refined and focused to capture complex situations.
  • Does the alert draw the eye without being visually overwhelming or annoying? Here’s a article about how to “reduce visual noise” in dashboards.
  • Is color used appropriately? Red means bad. Yellow is sorta bad. Green means good (but “good” things don’t need to be alerts). It isn’t particularly fair for color blind folks, but these conventions are deeply rooted.
  • Have you found the best mechanism for presenting alerts? Alerts can be sent through e-mail, as SMS message, blasted over the office intercom system, or posted to the wall in the bathroom. What is the most convenient and appropriate medium?

Control: Advanced alert system should give users the ability to customize and manage alerts.

  • Can the user identify the important alerts for them, and avoid the others? As hard as you may try in designing the dashboard or report, you aren’t in the shoes of the users. They will learn what they want to pay attention to and what information is extraneous.
  • Can the user adjust the parameters? With more sophisticated dashboards, you want to give users the ability to adjust parameters to hone in on the exceptions that really require action.
  • Can the user analyze alert frequency and trends? I’ve never seen a system that does this, but having the ability to view and analyze alert history seems critically important to getting a holistic view of performance.

0 comments | Add a comment

Your name

Email (optional, will not be shared)

Type the word "juice" (required to confuse the spammers)

Your comment






Real-World Tufte Graphics in 11 Lines of Code

Check out our followup post that describes how we created a downloadable Windows application or an excel spreadsheet you can use to create these graphics.

One of the troubles with Tufte is the frustrating infeasability of his approach to design for real people in business. One of his recommendations is to use Adobe Illustrator.

Adobe Illustrator is a big serious program that can do almost anything on the visual field (other than Photoshop an image). Most of my sparkline work was done in Illustrator. Fortunately all graphic designers and graphic design students have the program and know how to use it, so find a colleague who knows about graphic design.

Raise your hand if you have a graphic design assistant at your beck and call. I thought not.

One of the tools we use for rapid prototyping at Juice is NodeBox.

NodeBox is a Mac OS X application that lets you create 2D visuals (static, animated or interactive) using Python programming code and export them as a PDF or a QuickTime movie. NodeBox is free and well-documented.

All true. But it's more helpful to think of NodeBox as a free Adobe Illustrator that you can program in the world's easiest programming language. Oops, here's the right link.

I wanted to see if we could reproduce the following graph from The Visual Display of Quantitative Information, p 158.

Tufte Current Receipts Graphic

Here's the code. It's 11 lines of code if you exclude entering the data and setting things like fonts and colors.

size(500,700)
font('Palatino'); 
fontsize(12)  
stroke(0.4)  # a medium grey for lines
fill(0.2)    # a slightly darker grey for text  

<h1>data = (label, first, last, label-fudge-factor)</h1>

data = [ ('Sweden', 46.9, 57.4, 0., 0.),
         ('Netherlands', 44.0, 55.8, .3, 0.),
         ('Norway', 43.5, 52.2, 0., 0.),
         ('Britain', 40.7, 39.0, 0., 0.),
         ('France', 39.0, 43.4, 0., 0.6),
         ('Germany', 37.5, 42.9, 0., -0.4),
         ('Belgium', 35.2, 43.2, 0., 0.),
         ('Canada', 35.2, 35.8, .8, 0.4),
         ('Finland', 34.9, 38.2, -0.5, 0.),
         ('Italy', 30.4, 35.7, 0.3, -0.3),
         ('United States', 30.3, 32.5, -0.3, 0.),
         ('Greece', 26.8, 30.6, 0.4, 0.),
         ('Switzerland', 26.5, 33.2, -0.2, 0.1),
         ('Spain', 22.5, 27.1, 0., 0.3),
         ('Japan', 20.7, 26.6, 0., 0.), ]

text("Current Receipts of Goverment as a Percentage of "
      "Gross Domestic Product, 1970 and 1979", 20, 70, width=215)
text("1970", WIDTH*.28, HEIGHT*0.03)
text("1979", WIDTH*.68, HEIGHT*0.03)

def ypos(val):
    # calculate a vertical position by scaling between 10% and 90% 
    # of the height of the image
    return HEIGHT * (0.9 - 0.8 * (val - minval) / (maxval - minval))

<h1>find the minimum and maximum values in the range</h1>

alldata = [d[1] for d in data] + [d[2] for d in data]
minval, maxval = min(alldata), max(alldata)

for label, start, end, startfudge, endfudge in data:
    align(RIGHT)
    text(label, 0, ypos(start+startfudge)+4, width=0.25*WIDTH)
    text("%0.1f" % start, 0.25*WIDTH, ypos(start+startfudge)+4, width=0.07*WIDTH)
    align(LEFT)
    text(label, WIDTH*.75, ypos(end+endfudge)+4)
    text("%0.1f" % end, 0.68*WIDTH, ypos(end+endfudge)+4, width=0.07*WIDTH)
    line(WIDTH*.33, ypos(start), WIDTH*.67, ypos(end))

Here's what the result looks like.

Tufte Current Receipts Graphic with NodeBox

We have some great followups to this planned for next week. We'll reimplement this code with the Python Imaging Library, which will open things up for Windows users. We have some great plans for mashing these graphics up with our just released Google Analytics API.

Check out our followup post that describes how we created a downloadable Windows application you can use to create these graphics.

21 comments | Show all comments only the last 5 are shown


May 16, 2008
Chris Gemignani said:

Who's up for a multi-language infographics shootout?


May 18, 2008
Tim said:

That's cool !

I was wondering if there was a way to generate these graphics through command line ? that way we could embed this in web application and get the graphics generated dynamically

note: looks like comments in your code got converted to html (# -> h1)


May 18, 2008
Kragen Javier Sitaker said:

Is there a way to get old-style numerals with NodeBox? I suppose you have to find an installed font on your Mac with old-style numerals.

Pradeep's processing.js demo is awesome, but from the screenshot lacks antialiasing. (I'm not yet a Firefox 3 Achiever.)


May 19, 2008
Luke said:

Dude, why reproduce the errors ("fudge factors") in the original?


May 26, 2008
The Dude said:

@Luke: Dude, the fudge factors are not errors. They are there so that the text labels do not overlap.

Your name

Email (optional, will not be shared)

Type the word "juice" (required to confuse the spammers)

Your comment


Add a comment





S. Few Renounces Dual-Axis Graphs; Juice Ups Ante

After deep introspection, Stephen Few has determined that graphs with dual-scaled axes are fundamentally flawed. Rather than risk the potential for confusion, he believes that there are superior graphing approaches for situations where related data series have different units or magnitudes. His measured and thorough analysis concludes:

“It is inappropriate to use more than one quantitative scale on a single axis, because, to some degree, this encourages people to compare magnitudes of values between them, but this is meaningless.”

I commend Stephen for the courage to start down this path, but he hasn’t gone far enough. Here at Juice, we must often take controversial positions. You may remember that we were among the first to criticize Microsoft’s “databars”, the first to take on the powerful Dashboard Gauge lobby, and the first to challenge the applicability of Tom Davenport’s “Competing on Analytics” sales machine.

While it is true that the second axis can be deceptive, let’s not let the first axis off without asking some tough questions. It is the confusion—nay, the collusion—of the two that causes trouble—who is to say which is the bad seed? We must ask ourselves, do not axes belong in the “Axis of Evil”?

The problem is broader than Stephen suggests: axes are just the tip of the iceberg when it comes to graphic bling that potentially distract or confuse readers:

Take data labels, for example. They encourage users to consider specific values rather than focusing on relative sizes or placement of graph lines or bars.

Legends draw the reader’s eye away from the central storyline of a graphic.

Gridlines… please don’t waste my time with these flat faux-series. One wouldn’t put pinstripping on a Ferrari.

Place your graph in proper context and titles become redundant.

Minimalism is in. Extraneous graph decoration is out. Look no further than Tufte’s sparkline: no excessive graph decoration there.

sparkline

The world cries out for a new charting aesthetic. One that champions elegance and casts down gaudiness. Let us evoke the pure visual essence of the data. Let us find a pure form to evoke the emotion and hidden meaning of the data. Now is the time for Naked graphs—stripped to the essentials (TM).

Our argument is simple: the visualization of information is the message. The data is but an intermediary form of that visualization. Therefore, any residue from the raw data should be scrubbed from your final graph. Only when you achieve this unadulterated state will the meaning of the graphic burn its way into your consciousness.

Here’s an example of an analysis that casts light on both the relationship of the Fed to hedge funds while simultaneously answering your question about what happened with last month’s sales in the Newark division.

naked analysis

Truly here we see the words of Mark 9:43 made real:

If your hand causes you to stumble, cut it off; it is better for you to enter life crippled, than, having your two hands, to go into hell, into the unquenchable fire.

Gaze in awe, viewers, and find wisdom on this very foolish day.

8 comments | Show all comments only the last 5 are shown


April 12, 2008
Jeff said:

A S. Few article reference, the ever gratuitous Tufte mention AND a verse from the bible - all in one article, talk about data density...


April 12, 2008
dave said:

I generally agree with your philosophy of minimizing "chart junk" but I think you may be going to the extreme here.

Most consumers of chart information are not analytics professionals and need help to interpret.

- Data labels: my users demand them. They want to know the value. I don’t think they have a negative impact on “… than focusing on relative sizes or placement of graph lines or bars”

- Legends: are you kidding? Really?

- Gridlines: absolutely necessary for bar charts. I won’t speak for you, but gridlines help my brain orient the chart information.

- Titles: Chart titles and axis labels are necessary and not at all distracting.

Is there not a comfy middle ground here?

| ^
| ^
| ^
| ^
| ^
| ^
| ^^
| ^ ^
| ^
_____________________________________

Dave


April 12, 2008
derek said:

Dave, check the date that article was posted :-)


April 13, 2008
dave said:

I'm a moron...

thanks derek!


April 18, 2008
tao said:

The root of the problem with visualization is that you are using an organ that is simply not meant to understand so much. The only solution to fully understanding data is to not visualize at all but use a direct neural implant into the brain that allows you to quickly grasp all aspects of the data. All this visualization and introspection about visualization is just trying to improve the horse buggy when the automobiles are coming.

Your name

Email (optional, will not be shared)

Type the word "juice" (required to confuse the spammers)

Your comment


Add a comment





Analytics Roundup: Better presentations

Feltron Eight
Nice example of an optional, but much more interesting annual report could be formatted.

The Steve Jobs 90 Minute Keynote in 60 Seconds - Mahalo
GREAT summarization of the Job MacWorld 2008 keynote—from 90 minutes to 60 seconds. This demonstrates a good example of how to summarize a great deal of information.

Presentation Zen: 6 Presentation tips from a Steve Jobs keynote
6 good presentation points based on Steve Jobs 2008 MacWorld keynote

Meryl.net » 70+ PowerPoint and Presentation Resources and Great Examples
Great examples of properly using PowerPoint/Keynote.

0 comments | Add a comment

Your name

Email (optional, will not be shared)

Type the word "juice" (required to confuse the spammers)

Your comment






Analytics Roundup: Google goodies v. MS Paint

YouTube—MS Paint by freeloveforum
Ah—MS Paint. The endless limitations. This spoof video pokes fun at the design team who made this application.

YouTube—Amazing Footage of MS Paint
Completely amazing step by step footage of the creation of a great image with (you won't believe it) MS Paint - no kidding. This just goes to show that so many times it's not the tool that enables or limits, but the skills of the user.

ColorSchemer | Instant color schemes for your Mac with ColorSchemer Studio OSX
Mac tool to properly select colors that look good together. Adds a new tool to base the scheme on a photo as well.

Amazon.com: Visualizing Data: Books: Ben Fry
Ben Fry is good.

google ridefinder
Shows paths of shuttles in New York City. It's easy to pick up the outline of Manhattan.

daily FedEx plane network
Animation showing FedEx flight patterns over a 2 day period. It's easy to visually pick out patterns from this (i.e. there's no doubt where Memphis is).

Google new chart API
URL to plot charts and return the result as an image right in the browser.

0 comments | Add a comment

Your name

Email (optional, will not be shared)

Type the word "juice" (required to confuse the spammers)

Your comment






Earlier writing