TECHNOLOGY: Big data, big limitations

IN the late 1990s, when the first dotcom boom was on the cusp of becoming the dotcom bust, a Malaysian deputy minister came out with a brilliant, albeit grammatically-challenged, observation.

This was during a time when many companies all over the world thought that just setting up a website or putting “dotcom” in their name made them “e-businesses”, and that this would guarantee their success.

The reality did not turn out that way, with many going broke, which had led the deputy minister to announce – embarrassingly, at a major business conference – that, “E-commerce is not a sales magic.”

Journalists covering the technology scene knew what he meant though. Too much venture capital (VC) money had been thrown about, and too many companies had invested in sparkly new technology buzzwords with no idea of what these technologies entailed – their actual promise and their real limitations.

In essence, too many companies had been too easily seduced by PowerPoint huzzahs, but lacked the Excel discipline required to actually create a viable business.

It’s happening again with a slew of technologies – artificial intelligence (AI), robotics, facial recognition and the like: all interesting, promising technologies whose potential is generally overstated.

The reality is actually like too many a Facebook relationship status: It’s complicated.

Take big data and its related fields: big data analytics and data science.

Shorn of its gimmickry, big data is, in essence, looking at and analysing unstructured data – not just those data residing in the neat rows and columns of yesterday’s relational databases. It recognises the fact that there is data, data everywhere, nary a datum to process.

There are enough real-world examples of it working, and not just in the business world. Data science has even been credited with playing a key role (subscription required) in ending Liverpool FC’s 30-year wait (and 30 years of hurt) for a Premiership title.

But just like Liverpool’s use of data science would have come to naught if the club didn’t have a tactically-astute manager, a brilliant squad, and some strokes of good fortune, big data is not going to transform your business if you don’t put the other pieces in place.

Big data gives you some insight. It allows you to make and test hypotheses, measure these outcomes, tweak and amend your hypotheses, and test them again. It allows you to explore.

It doesn’t work if you ignore your findings. It doesn’t work if you neglect making structural changes to your business to accommodate your findings. And it certainly doesn’t work if you depend on big data only.

Indeed, too much pressure is being piled on big data teams in too many businesses. They’re expected to do the job that other departments are supposed to do, instead of merely supporting them. They’re being held liable for management malpractice, when the C-suite does not practise what it preaches.

They’re expected to work miracles.

But, you know, that’s not how it works. Big data can work wonders. But it is not “a sales magic”.