We frequently get requests to review and write about analytics-related solutions. I’ve put off most of these requests because it sounded like a lot of work. Then I had an 4-hour-work-week-style epiphany: most new users only give a new product or service a few minutes before they make up their mind. Why can’t I make the same snap judgement and call it an expert opinion?

First up is Open Flash Charts, pointed out to us by Matt Bear. This is an open source project started by John Glazebrook to provide flash charts that can be embedded in web pages. I love John’s explanation for taking on this project:

“Once upon a time I had to deal with a company who sell flash charting components, their component had a bug that I needed fixing, so I emailed them about it asking when it’d be fixed. (Remember that I had paid real money for this software.) They were so incompetent, rude and obnoxious that after three or four weeks of emails I thought to myself “I could learn Flash and Actionscript and write my own charting component, release it as Open Source, host it on sourceforge and build up a community of helpful coders faster than they can fix a single bug.” And that is what I did. And that is why it is free. I guess the moral of the lesson is: don’t piss off your customers.”

Great lesson. Great attitude. There are a bunch of vendors in this space (Fusion Charts, AnyChart, ILOG, PHP/SWF Charts, amCharts, Corda) and the going price seems to start at $500 for a developer's license up to $5,000 for an enterprise license. (Apparently that doesn’t always come with customer service.)

Open Flash Charts isn’t as flashy as any of these products, but that tends to be a good thing for charting components. Here’s a column chart from Fusion charts (notice how each bar is a separate color, for no good reason)

Here’s the Open Flash Charts

Open Flash Charts does a number of things well:

• It seems to be easy to implement. Basically, you just copy the Open Flash Chart SWF file into your web server, then start embedding flash charts into your HTML and point to either static or dynamic data on your server.
• You can configure data labels, background, number formats, on-click events, tooltips, etc.
• All the basic chart types are available (bar, line, area, pie, scatter).
• The help forum seem both lively (multiple messages a day) and supportive (a generally polite tone with lots of code posted).

On the negative side, Open Flash Charts doesn’t totally succeed in terms of data visualization fundamentals. The default charts have some contrast issues, odd color choices, and a little excess chartjunk. And when the charts get some “pizzazz,” things get worse:

I know… it is an open source project, so I should step up and fix the things I don’t like. I would, but I just ran out of my 10 minutes.

There is a tendency with reporting, and dashboards in particular, to cram as much information on the page as possible. It is a problem that Avinash describes with typical candor:

“This one of the core reasons why most dashboards are 'crappy', i.e. they are data pukes that provide little in terms of context and even less in terms of actionable value.”

In the past, we have offered tools to make data presentation as clear as possible (chart chooser, Excel chart cleaner). Sometimes clean isn’t enough; a more dramatic approach is needed.

One alternative is to shift the focus from the full data to changes in the most critical data points. By pulling out the important exceptions, you can make it easier for your audience to digest what matters and take action.

Stephen Few says in his book Information Dashboard Design:

“The best way to condense a broad spectrum of information to fit onto a dashboard is in the form of summaries and exceptions…given the purpose of a dashboard to help people monitor what’s going on, much of the information it presents is necessary only when something unusual is happening; something that falls outside the realm of normality, into the realm of problems and opportunities. Why make someone wade through hundreds of values when only one or two require attention? We call these critical values exceptions.”

Alerts are one mechanism to turn the focus to the exceptions, outliers and data highlights. Whether embedded in the dashboard or presented separately, alerts can be the extra layer of abstraction that make a dashboard useful. Unfortunately, they are hard to get right. I’ve arrived at four C’s for effective alerts—context, cogency, communication, control. Here’s a checklist to consider as you build alerts into a dashboard or report:

Context: Users need to understand how an alert is defined and how it fits into the larger picture.

• Are the parameters well defined? An alert is commonly defined by the following factors: metric (e.g. revenue), dimension (e.g. time), delta (e.g month over month change), scope (e.g. Northeast region, Peanut-product line), threshold (e.g. increase or decrease of 10%).
• Is the timing of the alerts actionable? One client explained to us that fluctuations in many of their metrics make monthly alerts too frequent—it would unnecessarily alarm people when, from their perspective, no significant trend had been established.
• Is the change statistically significant? This is of particular importance when you are measuring deltas. A doubling of traffic from a referring site doesn’t mean much when it is moving from one to two visitors.

Cogency: An alerting system needs to avoid causing unnecessary alarm while delivering easy-to-understand information that can be acted upon.

• Can the alerts be described in simple terms that even an executive can understand? Alerts should have a real-world meaning that users are familiar with. If an alert is based on a complex metric, for example, users will be confused as to the implications.
• Is the alert actionable? In the best cases, alerts should point users to both the drivers of the alert and the actions that can address the situation. This system does neither: ![terror warning system]
• Are the alerts so granular and/or frequently triggered that users will get alert fatigue? Excessive use of alerts will undermining their credibility. We saw this happen at one client where an IT-designed system threw off alerts like they were going out of style. The application went out of style the next year when users decided it was more distracting than useful. Here’s another example of a system that seems designed to raise blood pressure.

(It appears that a 5% increase in brand attribute performance isn’t good enough to get you out of the yellow.)

Communication: Alerts must be designed to effectively capture attention and inform.

• Is the alert placed in context? Google Finance does a nice job of putting news alerts within the stock chart.
• Is it clear what the user should do next? Give the user a clear path to more information so they can understand the full context of the alert.
• Does the sophistication of your alerts match the sophistication of your audience? I’ve found that it is better to start with some simple alerts so your audience can begin to learn what they mean and how to react. Over time, these alerts can become more refined and focused to capture complex situations.
• Does the alert draw the eye without being visually overwhelming or annoying? Here’s a article about how to “reduce visual noise” in dashboards.
• Is color used appropriately? Red means bad. Yellow is sorta bad. Green means good (but “good” things don’t need to be alerts). It isn’t particularly fair for color blind folks, but these conventions are deeply rooted.
• Have you found the best mechanism for presenting alerts? Alerts can be sent through e-mail, as SMS message, blasted over the office intercom system, or posted to the wall in the bathroom. What is the most convenient and appropriate medium?

Control: Advanced alert system should give users the ability to customize and manage alerts.

• Can the user identify the important alerts for them, and avoid the others? As hard as you may try in designing the dashboard or report, you aren’t in the shoes of the users. They will learn what they want to pay attention to and what information is extraneous.
• Can the user adjust the parameters? With more sophisticated dashboards, you want to give users the ability to adjust parameters to hone in on the exceptions that really require action.
• Can the user analyze alert frequency and trends? I’ve never seen a system that does this, but having the ability to view and analyze alert history seems critically important to getting a holistic view of performance.

Last week, we shared a rendition of a Tufte graphic using just a few lines of Nodebox code. As our commenters pointed out, Python is great, but it may not be every business analyst's carnal desire to learn a programming language just to generate some nifty graphs. I spent some time to push Chris's Nodebox rendition into a PIL-based Windows tool that can generate the same sort of comparison graph from an Excel file on the fly.

The result is The Comparison Chart Generator 1.0. The installation instructions are relatively simple. Unzip the zip file, and run comparisionchartgenerator.exe.

Alternatively, we have a new excel chart that creates the same effect using only excel functionality. Download the Excel Tufte Line Chart here.

If you are using the Chart Generator, start with some data in an Excel (xls) or Comma Delimited (csv) format. The data for this graph has to be contained within the first sheet starting with cell A1, as in the following picture.

Select an input file. There are a couple example files bundled with the download.

After selecting a file, you'll be prompted to modify a few of the basic options available for the chart.

Finally, save the result as a jpeg.

Here is the same image found in Tufte's textbook processed using the Comparison Chart Generator. It is generated using the csv example file bundled with the download.

Those of us who have undergone lasik eye-improvement surgery may still prefer the sharp crisp Nodebox results, but for the rest of us, this image looks pretty good. Let us know if this tool is useful. If there is enough of a positive response, we may consider expanding functionality for other fancy Tufte-esque charts.

If you do prefer Nodebox, I have an updated script here. This pushes the script up to 20 lines of code or so, but the extra 9 lines allow the labels to push themselves apart on their own. If you want to look at the source code for the Windows program, you can get it here. I used py2exe to compile it into an executable. The code, however, has not been thoroughly commented or cleaned as of yet, so edit it at your own risk.

Check out our followup post that describes how we created a downloadable Windows application or an excel spreadsheet you can use to create these graphics.

One of the troubles with Tufte is the frustrating infeasability of his approach to design for real people in business. One of his recommendations is to use Adobe Illustrator.

Adobe Illustrator is a big serious program that can do almost anything on the visual field (other than Photoshop an image). Most of my sparkline work was done in Illustrator. Fortunately all graphic designers and graphic design students have the program and know how to use it, so find a colleague who knows about graphic design.

Raise your hand if you have a graphic design assistant at your beck and call. I thought not.

One of the tools we use for rapid prototyping at Juice is NodeBox.

NodeBox is a Mac OS X application that lets you create 2D visuals (static, animated or interactive) using Python programming code and export them as a PDF or a QuickTime movie. NodeBox is free and well-documented.

All true. But it's more helpful to think of NodeBox as a free Adobe Illustrator that you can program in the world's easiest programming language. Oops, here's the right link.

I wanted to see if we could reproduce the following graph from The Visual Display of Quantitative Information, p 158.

Here's the code. It's 11 lines of code if you exclude entering the data and setting things like fonts and colors.

size(500,700)
font('Palatino'); 
fontsize(12)  
stroke(0.4)  # a medium grey for lines
fill(0.2)    # a slightly darker grey for text  

<h1>data = (label, first, last, label-fudge-factor)</h1>

data = [ ('Sweden', 46.9, 57.4, 0., 0.),
         ('Netherlands', 44.0, 55.8, .3, 0.),
         ('Norway', 43.5, 52.2, 0., 0.),
         ('Britain', 40.7, 39.0, 0., 0.),
         ('France', 39.0, 43.4, 0., 0.6),
         ('Germany', 37.5, 42.9, 0., -0.4),
         ('Belgium', 35.2, 43.2, 0., 0.),
         ('Canada', 35.2, 35.8, .8, 0.4),
         ('Finland', 34.9, 38.2, -0.5, 0.),
         ('Italy', 30.4, 35.7, 0.3, -0.3),
         ('United States', 30.3, 32.5, -0.3, 0.),
         ('Greece', 26.8, 30.6, 0.4, 0.),
         ('Switzerland', 26.5, 33.2, -0.2, 0.1),
         ('Spain', 22.5, 27.1, 0., 0.3),
         ('Japan', 20.7, 26.6, 0., 0.), ]

text("Current Receipts of Goverment as a Percentage of "
      "Gross Domestic Product, 1970 and 1979", 20, 70, width=215)
text("1970", WIDTH*.28, HEIGHT*0.03)
text("1979", WIDTH*.68, HEIGHT*0.03)

def ypos(val):
    # calculate a vertical position by scaling between 10% and 90% 
    # of the height of the image
    return HEIGHT * (0.9 - 0.8 * (val - minval) / (maxval - minval))

<h1>find the minimum and maximum values in the range</h1>

alldata = [d[1] for d in data] + [d[2] for d in data]
minval, maxval = min(alldata), max(alldata)

for label, start, end, startfudge, endfudge in data:
    align(RIGHT)
    text(label, 0, ypos(start+startfudge)+4, width=0.25*WIDTH)
    text("%0.1f" % start, 0.25*WIDTH, ypos(start+startfudge)+4, width=0.07*WIDTH)
    align(LEFT)
    text(label, WIDTH*.75, ypos(end+endfudge)+4)
    text("%0.1f" % end, 0.68*WIDTH, ypos(end+endfudge)+4, width=0.07*WIDTH)
    line(WIDTH*.33, ypos(start), WIDTH*.67, ypos(end))

Here's what the result looks like.

We have some great followups to this planned for next week. We'll reimplement this code with the Python Imaging Library, which will open things up for Windows users. We have some great plans for mashing these graphics up with our just released Google Analytics API.

Check out our followup post that describes how we created a downloadable Windows application you can use to create these graphics.