Will Titanic Data Hit an Iceberg?


I had a fun conversation on Twitter with Sam Perrin (@SamPerrinND) who is with Naked Data in Melbourne, Australia. I had a heck of a time finding their website, since Google helpfully filtered out the word “Naked” from my search to make it work safe. (That’s a whole different blog post, there.) The playful conversation led to a far more serious consideration of ethics and trust issues in the big data analytics and cloud movements.

It all started with me re-tweeting a link to “Big Data Will Be Just ‘Data’ as Technologies Mature by 2016: Gartner,” a little article on Computer Business Review relating to one of Gartner’s predictions regarding big data. “According to Gartner, big data will grow beyond its hype towards 2016 to become just data once the technologies mature, and organisations learn how to deal with it.”

titanic 1

The Twitter conversation went something like this:

SamPerrinND: @RobertsPaige @ActianCorp @gartner Already is! With data growing so quickly, there’s only so many adjectives we can come up with!

RobertsPaige: @samperrinnd Big data led to huge data, tremendous data, gargantuan data and then we got to the amazing future of ginormous data. ;-)

SamPerrinND: @RobertsPaige I look forward to the 2019 Titanic Data Conference.

RobertsPaige: @samperrinnd Let’s just hope there aren’t any icebergs. ;-)

SamPerrinND: @RobertsPaige Well played.

That, of course, got me to thinking about what sort of iceberg could possibly sink the big data movement. It’s got such tremendous momentum behind it at this point, I would think it would be unsinkable. But that kind of thinking does not have the best historical track record.

SamPerrinND: Great piece by @timoelliott in light of the NSA debacle. @RobertsPaige, speaking of icebergs… – (Link to this piece on the NSA scandal: The Ethics of Big Data: Vendors Should Take a Stand, which used an iceberg as a metaphor for big data.)

That included a definition of big data analytics that I hadn’t heard before – “using data that was previously ignored because of technology limitations.” That certainly plays into the Gartner prediction that once the technology matures, the “big data” concept will fade and become just another aspect of data management.

I had just read a piece by David Linthicum, shaking a finger at the NSA for blocking progress in the cloud computing movement by making it even more difficult for people and corporations to trust that the privacy of data on public clouds will be respected.

RobertsPaige: @SamPerrinND More an iceberg for cloud than big data analytics, I think. @DavidLinthicum seemed to think so: (Link to: Thanks, NSA, you’re killing the cloud: The current NSA scandal raises a ton of questions – and gives enterprises another excuse to resist the cloud)


Right after I tweeted that response, though, I reconsidered. Cloud and big data are intertwined in many ways, both technologically and culturally. What casts doubt on the ethics and safety of one, affects the perception of the other. We’re all riding on the same ship.

While big data analytics may progress to the point where it becomes as ubiquitous as more conventional forms of business intelligence, the lifeblood of that movement is data. If people, as a whole, lose trust in the ways their data are being stored and used, and withdraw from using such services, that could sink the whole thing cold.

I’m not sure if it makes me more cynical or less cynical, but I don’t believe that will happen. People’s trust has been damaged, but how many will give up their cell phones because of it? This ship is powered by business and personal advantage, and big data analytics offers tremendous leaps forward in every aspect of life and business. The data will still be put out there and collected. The question becomes, how will we, as a society, allow it to be used? Rather than manning the lifeboats at this point, I think we need to look a little more closely at who is steering the ship.

About Paige Roberts

As Actian’s Hadoop Analytics Evangelist, Paige identifies innovative big data and analytics trends and explores technology alternatives to help organizations drive actionable business value from their data. A seasoned software industry veteran with more than 15 years’ experience, Paige has worn a variety of hats including engineering, consulting, marketing and training. Follow her at @RobertsPaige.

View all posts by Paige Roberts →