Big Data Caution…from GK Chesterton

“The real trouble with this world of ours is not that it is an unreasonable world, nor even that it is a reasonable one. The commonest kind of trouble is that it is nearly reasonable, but not quite. Life is not an illogicality; yet it is a trap for logicians. It looks just a little more mathematical and regular than it is; its exactitude is obvious, but its inexactitude is hidden; its wildness lies in wait.”
…GK Chesterton, “Orthodoxy”

It was with great interest I ran across this comment the other day. And it got me thinking about the world of big data today.

The red flag for data abuse comes when people cede their human initiative and let data take over. Listen to how people discuss “big data” and you’ll start getting a sense their vision is to have data run the world. I suppose in a corporate bureaucracy this provides perfect cover for a mistake. (“The data said to do it” or perhaps “The Data Scientists said it would work!”).
Read more of this post

The Search for Meaning in Big Data

Big Data has been around long enough now that it’s clear that analyzing big data can deliver some sweet benefits. What’s less discussed are the challenges companies face in finding those benefits.

A Key Challenge:  Irrelevant Data. My friend Shahin Khan recently tweeted about one of the key challenges with big data.

@ShahinKhan The ratio of relevant data to irrelevant data will asymptotically approach zero. #BigData

Read more of this post

Big Data. Big Promise. Big Caution.

Big Data imageBig data claims to be the new salvation for all businesses. Because, we’re told, big data will discover amazing new truths. Time will tell.

But in the meantime, most big promises should also be accompanied by big cautions. Which one’s are most important as we approach big data? Recently, on the Financial Times website, Tim Harford wrote a blog post on the topic: Big Data: are we making a big mistake. It is one of the few really thoughtful big data discussions we’ve come across in a while.
Read more of this post