That was a great talk! I really liked the slides, the pictures were great and there were some very quotable statements. I think especially the points about reporting vs. analysis and data/analysis vs. presentation were good and are often overlooked. The remarks about BI being a social rather than a technical problem, and about "chart-based encryption" were also spot on: it's unbelievable what visual nonsense is being produced.

Your examples were also interesting, because they were unexpected. NameVoyager and WeFeelFine.org aren't exactly what I would consider business visualizations. But they clearly illustrate what good visualization should do (though they are certainly more on the presentation than the analysis side, which also makes them so appealing: they tell a story, or at least give you the building blocks for one).

What I missed was some kind of big final example at the end that would apply what you said to some real business cases. I am all for visualizing blogs and baby names, but how do these things apply to the typical BI or CRM application? And what does analysis vs. presentation mean in a practical example?

I can't understand business these days.

Microsoft expanded the graphic "blings" in Excel, because business asked for all these realy wierd pyramid columns in pastels. Pastels!! Business asking for pastel colors for presentations??? I thought presentations were to upset the audience, not put them to sleep.

What's the relevance of cows? Is it related to a new advertisong campaign? Analysis is how you can distort the facts to pursuade the audience.If you have to wake up the audience with spectacular displays, why not just bring a wide screen HD tv set and do a surroung presentation with cows mooing in the background (7.1 sound).

You could perhaps have a parade around the perimeter walls of the conference room, with a real brass band. Get everybody to join in. Signifies agrrement with the latest management program.

DAHeiser

The slides and discussion really resonated with me. For 10 years I I labored with the presentation of 1,000s of slides to execs about customers and statistics at The Coca-Cola Company, Thursday's discussion finally answered my question, "what am I doing wrong?" The problem was or is a social one. My perspective is changed.

Thank you for providing the enlightenment.

Guillet

Zach:

Good job.

I'm shamelessly thieving your approach with your PowerPoints and dropped $11 on iStockPhoto to get the appropriately resonating photos for an upcoming conference presentation.

The "chart-based encryption" joke was made funnier since you had no way of knowing if any of the webinar audience "got it." As an East Coaster transplanted west, I've had my share of lines during talks that were pure comedic gold be met earnest non-comprehension . (Another reason we're so well liked...)

Great stuff; I've already dropped the herding cats vs. herding cows distinction on a co-worker.

BT

Please come and give this presentation at my company. I and my coworkers have given similar ones for years now, but they don't seem to carry the same weight as the ones that better-paid outside contractors give.

As an outsider you'll have instant credibility with my senior management, which I and my coworkers lack because we're part of - you know - a cost-center.

To ensure that everything you say to my superiors is taken as seriously as possible, I will disavow any knowledge of having made this post...

[...] I was interested to read what Zack at Juice Analytics had to say on the matter. To paraphrase, reporting is for things we know well and are predictable where as analysis if for things that are unknown and erratic. Now I’ve only looked at the slides (check them out there good!) so I may have missed the context, but that’s not really the point. [...]

interesting presentation. I am especially drawn to the Mindset|Skillset|Toolset paradigm and would be interested in the audio recording, an expansion of these ideas or maybe some reference to follow-up on.

I've always thought that if you provide the tools, the mindset will follow. Sometimes is does and and sometime there's a lot of kicking and screaming.

I'm in an org where the tools are there (multimillion dollar BI/Datamart/Reporting "solution", but the mindset is non-existent and I'm wondering how to overcome this.

The concept of BI as a social problem is compelling and I would like to know more about your thoughts on this.

Great article and I love the subtle humor along with the reference to Dr. Evil! The last paragraph made me lol.

It amazes me the price tag on this "super" computer. I mean, this could probably be done with MS Access...

On Deal or No Deal the other night, there was a teacher who made the comment that $40k was way more than what she made in a year. $80 million could be spent on underpaid teachers who we rely on to teach our kids who are the next generation's leaders!

Great Post!

This reminds me of an article I read a few years back on how the University I attended was spending millions on an anti-spam email system when most of the best products out there for this are free.

At a time when you can buy a dual processor (with dual cores) machine with 4+ Gigs of RAM and over a terabyte of disk for less than $10K, I wonder what the other $79,990,000 is being spent on. Licensing and Installation?

A couple of observations:

1) For that kind of money I would've demanded a much hipper acronym--Aris? At least military contractors tend to give very cool names to theWhite Elephant projects they fob off to the gov't.

2) And as usual, no one thinks through the anthropology. So I'm the geometry teacher who has my sh*t together and a solid track record that's now in the 'system'. How will I react when every other teacher in NYC makes their students struggling with geometry my problem????

BT

One more related article of interest : "ARIS Critics: IBM Responds"
http://nycpublicschoolparents.blogspot.com/2007/03/aris-critics-ibm-responds-march-7-2007.html

hmm 80.000.000 dollar for a database with one million students.
"Will crunch huge amounts of data and offer up-to-the-minute reports"
What, do those boys and girls get grades every minute or so?

With one million students there must be a student with a talent for MS access that can build it for 100 bucks. Shoot, you can almost vlookup it together in excel 2007! (not that you would want that of course).

Someone over there must really like IBM.

There seems to be a trend of education companies/districts being overcharged for data ware house and analytics. I wonder if NYC is being charged by the record.

Sorry, if you only had a million records that would have been $40 million. However, those extra 100k records are going to double the cost. You know... servers and processing costs are expensive!

This measurement and reporting system addresses a non issue in education.

The fundamental problem with education is that it fails to adopt what we have learned about quality control. Namely, you cannot inspect quality into a product. The emphasis on testing fails the to address that education is not simply a measured outcome, it is a process. I would suggest spending the money to address the questions of what the inputs (students) look like and a real focus on teaching (the process) and less on testing (measurement and reporting).

Toyota does not inspect quality into each car. Rather, they focus on ensuring the parts are at spec and that the process is rigorously followed (and continuously improved). In education we have hundreds of curriculum that are not really validated to produce the specified outcome. At best they are benchmarked against some control which itself is not linked to the targeted outcomes. We further aggregate the process by not taking care to know who we are really teaching. No, in the name of inclusiveness, we often seek a wide range of skills among students to make process management impossible. When will we learn?

Very interesting blog. I like bee's comments--sometimes we get too much savvy about technology instead of focusing on the core of the problem. The Toyota example is apt--we need to build quality into the process (of education), and not spend too many $$ on measuring (inspecting) the product (student).

Bill Gates, in front of the committee of education, labor, etc. emphasized upon merit-based pay for teachers--we need to spend money there. If there was more money in teaching a lot of smart people would enroll in the teaching profession (I would!).

Where did the idea that this was a no-bid contract come from? It was a multi-month open bidding process in which much of the education industry and enterprise technology industry competed. McKinsey (hardly a naive client and with their bill paid by a foundaiton) supported NYC in evaluating the bids. A supercomputer is not involved in any way.

I don't want to say anything I would regret on the Juiceblog, so I'll just say that its probably worse than you think, take a look at these powerpoint/pdfs I downloaded from http://schools.nyc.gov :

http://www.datawrangling.com/media/AchievementReportingandInnovationSystemARISPanelPP.ppt
http://www.datawrangling.com/media/001_092706OnlineDeck1.pdf

"ARIS COMBINES OLD SYSTEMS AND NEW CAPABILITIES"

"Stoplight" charts...oh boy.

Look at slide 18, "Overview of Required Functionality". They don't even ask for built-in data visualization (they will export to Excel), this is just an $80M content-management system. Everything they list as a requirement is 5 year old technology, which you could do with Drupal, Plone, or SugarCRM today out of the box. I get most of this with MoinMoin right now, or you could use Django if you want to make something really slick.

*Library of templates/idea bank for school-created tests
*Test creation and upload
*Assessment content upload into online testing, reporting and analysis
*Online testing and scoring
*Score data upload into reporting and analysis

*Canned and customized reports
*Drill-down, roll-up, filtering
*Ad-hoc queries
*User-generated databases
*Adding new data to central repository
*Data viewing in 3rd party tools (e.g., Excel)

Advanced analysis
(for selected users)
*Advanced statistics
*Pattern detection for target setting

*Library of templates for different document types (e.g., lesson plans, curriculums, )
*Document creation and upload to repository
*Document tagging
*Collaborative document creation
*Publishing approval
*Document linking to reporting (surveys)
*Searching documents in the repositories
*Document viewing and download
*Pattern analysis across text documents

*Posting announcements; setting alerts
*Online discussions; online messaging
*Custom workspaces, integration w/email

*Pre-defined home page based on user role
*Personalization of home page (HP) by user
*Adding new tools to HP from a tool catalog

Could it be the $80 Million should read $80 Thousand? Maybe it was reported as 80,000 and the reporter thought the figures were in x'000?