Organizations have a personality, and it bleeds into everything from executive reporting to product offerings. A recent Fortune article entitled Microsoft without Gates offers this wonderful tidbit about Steve Ballmer, CEO of Microsoft:

Even though he never was a serious computer programmer, by all accounts Ballmer is just as good at math as Gates is. He lives and breathes data. “Steve has a computer in his head,” says Bob Muglia, a 20-year company man who heads the Server and Tools division. Ballmer expects his subordinates to be adept in math as well. He distributes 11-by-17 sheets filled with numbers detailing the progress of various operations. The numerals are so small that executives use transparent magnifier rulers to see them. But there are never any columns showing percentage changes. Ballmer believes people ought to do that in their heads. It saves space on the paper for more numbers.

Wow. If it is as bad as the author describes, Ballmer has designed the anti-dashboard.

The Presentation Zen blog offers another great example of organization culture as displayed in business artifacts.

Gates here explaining the Live strategy. A lot of images and a lot of text...Good graphic design guides the viewer and has a clear hierarchy or order so that she knows where to look first, second, and so on. What is the communication priority of this visual? It must be the circle of clip art, but that does not help me much.

Does it get more "Zen" than this? "Visual-Zen Master," Steve Jobs, allows the screen to fade completely empty at appropriate, short moments while he tells his story.

A couple months ago, we put together a Greasemonkey tool that sucked data out of Google Analytics, and after mining it for trend information, integrated it back into the GA interface. This week's tool combines and extends Google Analytics with data from an outside source.

Here is a quick alpha of our Greasemonkey integration of external data reporting into Google Analytics for Kampyle, a "feedback analytics service." Click on the images to zoom in.

Clicking on the 'Kampylize' tab queries the Kampyle site in real-time to populate the standard GA data table.

Our friends at Kampyle run a service that allows website owners to put a feedback button on individual pages of their website. All information submitted by the user is uploaded to a central Kampyle database that compiles the user feedback with web page url and standard internet statistics such as the name of the browser. Website owners can access a server-end service that consists of a reporting site complete with summary data tables, graphs, and charts.

Since both sites are web-based reporting suites segmented in a similar fashion (individual website, date, web browser, etc.), they integrate together naturally. There is a lot of value in placing related data side by side, allowing users to get a more holistic picture of web site performance. If you have other ideas of data sources that would fit neatly with Google Analytics, let us know and we'll consider building the integration.

If you're interested in technical details, continue to Open Juice to see how this is all accomplished...

There is a tendency with reporting, and dashboards in particular, to cram as much information on the page as possible. It is a problem that Avinash describes with typical candor:

“This one of the core reasons why most dashboards are 'crappy', i.e. they are data pukes that provide little in terms of context and even less in terms of actionable value.”

In the past, we have offered tools to make data presentation as clear as possible (chart chooser, Excel chart cleaner). Sometimes clean isn’t enough; a more dramatic approach is needed.

One alternative is to shift the focus from the full data to changes in the most critical data points. By pulling out the important exceptions, you can make it easier for your audience to digest what matters and take action.

Stephen Few says in his book Information Dashboard Design:

“The best way to condense a broad spectrum of information to fit onto a dashboard is in the form of summaries and exceptions…given the purpose of a dashboard to help people monitor what’s going on, much of the information it presents is necessary only when something unusual is happening; something that falls outside the realm of normality, into the realm of problems and opportunities. Why make someone wade through hundreds of values when only one or two require attention? We call these critical values exceptions.”

Alerts are one mechanism to turn the focus to the exceptions, outliers and data highlights. Whether embedded in the dashboard or presented separately, alerts can be the extra layer of abstraction that make a dashboard useful. Unfortunately, they are hard to get right. I’ve arrived at four C’s for effective alerts—context, cogency, communication, control. Here’s a checklist to consider as you build alerts into a dashboard or report:

Context: Users need to understand how an alert is defined and how it fits into the larger picture.

• Are the parameters well defined? An alert is commonly defined by the following factors: metric (e.g. revenue), dimension (e.g. time), delta (e.g month over month change), scope (e.g. Northeast region, Peanut-product line), threshold (e.g. increase or decrease of 10%).
• Is the timing of the alerts actionable? One client explained to us that fluctuations in many of their metrics make monthly alerts too frequent—it would unnecessarily alarm people when, from their perspective, no significant trend had been established.
• Is the change statistically significant? This is of particular importance when you are measuring deltas. A doubling of traffic from a referring site doesn’t mean much when it is moving from one to two visitors.

Cogency: An alerting system needs to avoid causing unnecessary alarm while delivering easy-to-understand information that can be acted upon.

• Can the alerts be described in simple terms that even an executive can understand? Alerts should have a real-world meaning that users are familiar with. If an alert is based on a complex metric, for example, users will be confused as to the implications.
• Is the alert actionable? In the best cases, alerts should point users to both the drivers of the alert and the actions that can address the situation. This system does neither: ![terror warning system]
• Are the alerts so granular and/or frequently triggered that users will get alert fatigue? Excessive use of alerts will undermining their credibility. We saw this happen at one client where an IT-designed system threw off alerts like they were going out of style. The application went out of style the next year when users decided it was more distracting than useful. Here’s another example of a system that seems designed to raise blood pressure.

(It appears that a 5% increase in brand attribute performance isn’t good enough to get you out of the yellow.)

Communication: Alerts must be designed to effectively capture attention and inform.

• Is the alert placed in context? Google Finance does a nice job of putting news alerts within the stock chart.
• Is it clear what the user should do next? Give the user a clear path to more information so they can understand the full context of the alert.
• Does the sophistication of your alerts match the sophistication of your audience? I’ve found that it is better to start with some simple alerts so your audience can begin to learn what they mean and how to react. Over time, these alerts can become more refined and focused to capture complex situations.
• Does the alert draw the eye without being visually overwhelming or annoying? Here’s a article about how to “reduce visual noise” in dashboards.
• Is color used appropriately? Red means bad. Yellow is sorta bad. Green means good (but “good” things don’t need to be alerts). It isn’t particularly fair for color blind folks, but these conventions are deeply rooted.
• Have you found the best mechanism for presenting alerts? Alerts can be sent through e-mail, as SMS message, blasted over the office intercom system, or posted to the wall in the bathroom. What is the most convenient and appropriate medium?

Control: Advanced alert system should give users the ability to customize and manage alerts.

• Can the user identify the important alerts for them, and avoid the others? As hard as you may try in designing the dashboard or report, you aren’t in the shoes of the users. They will learn what they want to pay attention to and what information is extraneous.
• Can the user adjust the parameters? With more sophisticated dashboards, you want to give users the ability to adjust parameters to hone in on the exceptions that really require action.
• Can the user analyze alert frequency and trends? I’ve never seen a system that does this, but having the ability to view and analyze alert history seems critically important to getting a holistic view of performance.

Here’s a little predictive analytics:

About a year ago, I took a swipe at the “$80 million supercomputer to analyze NYC student achievement.” It smelled more like a super sales job than a super useful analytical tool.

At the time I had said:

Teachers are underpaid, hardly appreciated, and overworked. I can only wonder what the half-life is of a system that asks teachers to log on to get information delivered by the “chief accountability officer.”

Well, it appears that things haven’t gone that smoothly with the supercomputer. Today, I received a link from Leonie Haimson, a NYC education advocate, to a story entitled SCHOOLS COMPUTER AN $80M ‘DISASTER’.

Not only has the supercomputer struggled to gain much traction with users (“The school system’s new $80 million computer super system to track student performance has been a super debacle, teachers and principals say”), it has coincided with severe budget cuts.

We see these data warehousing problems all the time with our clients, and the NYC supercomputer displays all the hallmarks:

• Delivery delays: Nearly six months after the Department of Education unveiled the “first of its kind” data-management system, the city’s 80,000 teachers have yet to log on because of glitches and delays.


• Bad user experience: Many principals have complained that it runs slowly, lacks vital information, and is often too frustrating to use.


• Complicated training and set-up: School officials were hoping to have everyone hooked up and trained within months… delays in creating IDs and passwords for teachers
• Trying to do too much, delivering too little: The principal added that she preferred to get student information from a combination of old data systems “rather than wait for ARIS to churn and churn and churn and maybe give me half the report I need.”
• Massive cost: Complaints about the expensive system—on which nearly $35 million has been spent so far—have gotten louder since the city unceremoniously chopped $100 million from individual school budgets last month.
• And yet, few success anecdotes to justify the investment: ARIS had already enabled her data team to analyze the performance trends of the school’s many English-language learners.

It does offer one thing that I haven’t seen before: a Chief Accountability Officer.

There is little ‘r’ reporting and there is big ‘R’ Reporting, and the difference is vast:

reporting is the stuff that comes straight out of your reporting solution. It gets created by choosing a few parameters and typically shows up as a table of data with an accompanying chart.

Reporting is something altogether different. It is concise set of graphics and data that tell a focused story. It is crafted to focus on the key information and exclude everything else. It may come in the form of a single dashboard or a 20=page deck, but it is always audience-friendly. It is informed by context and provides explantation. Reporting is not about the numbers, it’s about what the numbers tell you.

By analogy, what if we didn’t make a distinction between a raw fish pulled out the sea and a prepared fish dinner? When the waiter slapped a still-squirming sea bass on my plate, I probably wouldn’t take much consolation in getting a deboning knife and a hot plate.

In the wild, the two species of reporting are often confused. To help you identify one from the other, I’ve put together a couple of examples with tell-tale signs:

The difference comes down to a gut-feeling: Was this document created to address the questions of a specific audience with a specific problem?

This may be a distinction that is implicitly well-known. My concern is more about explicit acknowledgement of the gap between them. And in the process:

1. Avoiding passing off reporting as Reporting. In particular, vendors who offer reporting tools think they are delivering the ability to communicate performance, when in fact they are mostly providing the raw materials.

2. Recognizing the level of effort required to transform reporting into Reporting. Analysts spend a huge amount of time filling this gap; it is one of the wasteful backwaters of modern enterprises.

This has been a common theme in my recent client discussions. People are sick of slogging through their reporting tools to build useful information for management. Ultimately, developing great Reporting requires an understanding of problems, the audience, and thoughtful design. But that doesn’t mean it should be so painful to construct. We are working on a solution to help, but in the meantime here are a few general things we do:

• Gather data in its cleanest form (CSV instead of heavily formatted XLS, or in the worst cases PDF)
• Automate data cleaning and manipulation steps using Excel macros and VBA
• Create repeatable and documented report building processes
• Try to convincing executives that less reporting can be more valuable