Design and Value of “Big Data”
Last Friday I had the opportunity to attend Web 2.0 Expo in San Francisco. San Francisco transit connections being what they are, I missed some of the opening keynotes, but luckily I didn’t miss Jeff Veen’s Designing for Big Data.
Jeff’s presentation echoed some of my thoughts — we have access to much more data than ever in the past — how can we gain value from it? As he noted, we don’t want to just “decorate the data.” We want to gain value from it. His new start up, Small Batch (apparently stealthy given this website), created Wikirank for visualizing and comparing what people are reading on Wikipedia. The tool provides interesting data in a clean interface with some user control in that they allow you to decide what data to compare:
Compare Wikipedia topics.
At the top of the front page, you’ll see a comparison of two or more topics that we’ve chosen. From that section, you can view more detail on the featured topics, share the chart with others, or choose your own terms to compare.
They also ask that you share particularly interesting comparisons with them (and provide an email link).
I think this last, user control of data comparisons, is going to be key in how we “design for big data,” and eventually in how we gain value from big data. We have the data. We are gaining the tools and skills. Next we need to understand how to extract value from this combination.
Quoting from a prior post:
Bellamy et al. draw from Card, Mackinlay, & Shneiderman’s book, “Readings in Information Visualization: Using Vision to Think” to suggest that visualizations should increase people’s visual capabilities and amplify cognition. They note that Card et al. outline six visualization benefits:
- Increasing the users’ memory and processing resources
- Reducing the search for information
- Enhancing pattern detection
- Enabling perceptual inference
- Using perceptual attention mechanisms for monitoring
- Encoding information in a malleable medium
I followed that post with one thinking about how teams might gain value from visualization of team data. Veen’s presentation, and his wonderful links back to data mavens like Edward Tufte, push me to think about how big data can provide more general value. Tom Davenport’s ideas around Competing on Analytics are a great background, but we have even better access to data inside and outside of our organizations since his 2007 book.
Here are some views into big data. I’m not saying they all have value, but provide them as a start to some brainstorming.
Wikirank (described above)
Yahoo Pipes allows normal people to “aggregate, manipulate, and mashup content from around the web”
ProgrammableWeb. If you want to go deep, go here.
What else should we add?
TweetTags: analytics, Davenport, small batch, Tufte, Veen, w2e, web2.0expo
6 Responses
April 25th, 2009 at 6:59 pm
Video of Jeff Veen’s talk available here: http://www.veen.com/jeff/archives/001000.html
April 28th, 2009 at 9:40 pm
[...] I’ve been following “big data” posts and articles — as well as pushing the opportunities provided by big data to my MBA students and anyone else who will listen (for example). [...]
April 29th, 2009 at 8:55 pm
[...] may be “medium data” (versus big data), but the same thinking applies. There is value in an overall focus on Evidence Based [...]
June 6th, 2009 at 5:35 pm
You should also look at http://www.wordle.net/ for a simple way to visualize data in the form of word clouds. Simply drop a bunch of text in and see what comes out. I’ve used it for my resume, ipod playlists, etc. but there are tons of ways this provides useful insights.
IBM also has a public visualization capability they call Many Eyes (http://manyeyes.alphaworks.ibm.com/manyeyes/) which allows one to visualize more complex data sets in different manners.
July 2nd, 2009 at 10:41 am
[...] community? Regardless, all of us are now in positions to make decisions about our use of “big data” and how we control our own [...]
August 1st, 2010 at 10:35 am
[...] can we use big data to make better management decisions? In earlier posts (here and here) I’ve listed a few tools and asked similar questions. Marissa described the power of other [...]