Big Data—bloated or just large-boned?

It appears that "Big Data" is the new rallying cry for consultants and others who can't quite explain what they're up to. So, at any conference worth its price tag, those who have been expounding ad infinitum about risk mitigation are often being out-shouted by those who see Big Data as the key to unlocking the heretofore insoluble mysteries of why supply chains operate the way they do.

But what is Big Data? Shoveling a path through the amassed piles of bovine excrement reveals a perhaps too-simple truth. Simply put, Big Data is more data than you know what to do with.

Said another way, Big Data is a pile of transactions and conditions that is just too humongous, and has too many dimensions, to be processed and analyzed by the traditional tools you have on hand. Last-century architectures resting on foundations of relational databases are not up to new-century tasks, volumes, and velocities or to the nuances in the data that are available today (with more on the way).

TRASH AND TREASURE
To defy those who would like to have a universal definition, it turns out that one company's trash is another's treasure. For a number of reasons.

One is raw data availability, in quantity and in quality, in refinement and in nuance, in relevance and in robustness. But each organization might well look at these radically different buckets of stuff, with enormously different potentials for analysis and the development of business intelligence, as Big Data.

Further, each company's tools for storage and analysis could be far apart in capacity and capability. The company with a really robust tool would, then, have a much different threshold for calling its source material Big Data than would the company with less sophisticated analytical capabilities, even if the data itself were virtually identical.

HOW WE GOT HERE
Much like the 800-pound person who can't be extricated from his home to be taken to a hospital, this is a process that takes time and starts small. It's the first binge on a bag of Cheetos that leads to snacking on gallons of ice cream.

Many of us are still recovering from the realization that a cheap wristwatch today has enough computing power and storage capacity to support a NASA Apollo mission. It is just too much for us to contemplate that the cloud (or buildings full of servers for the less-poetically inclined) permits data storage in quantities that we don't even know how to say. So, comfortable prefixes of "mega" and "giga" have given way to "tera" and "zebi." These new terms are, by definition, useful. If we did not have the technology to capture these quantities, we would have little need for names for them.

NOW THAT WE'VE ARRIVED
Not that we have reached an end point; we're only at one point in the continuing evolution of data acquisition, storage, and analysis. But we are at a point at which there is a more-than-nagging question, namely: What do we do with all this stuff?

The simple answer is that we analyze it; make up hypotheses; search for evidence in the data, pro or con; and make decisions. But there is a hard reality. Inordinately few supply chain practitioners know how to frame the questions, let alone how to tease relevant analytics out of the data, or—now it gets tricky—make seasoned, informed judgments from what Big Data seems to be trying to tell us.

Some, correctly in our view, have gone so far as to suggest that Big Data is pretty much pointless without Big Judgment. To complicate matters further, the point of Big Data analytics is **ital{not} to evaluate yesterday or model today; the information is considerably more useful when it can be manipulated to show us possibilities for the future. Adding a bent for predicting the future to the core requirement of an experience-based understanding of the data reduces the pool of capable and competent decision-makers even further—shifting perceptions of Big Data to being an interesting toy or the abstract province of impractical deep thinkers.

But for the moment, we need to face the reality that, in the collective, our ability to apply decent judgment to Little Data leaves a lot to be desired. We have some good analysts, but too many who cannot differentiate among the meanings of means and medians. And a distressing analytic population has a penchant for reaching universal conclusions from single observations (such as claiming that all inventories share movement and physical characteristics with auto parts).

SUCCESS AMONG THE DISAPPOINTMENTS
There are roses scattered among the thorns. A gigantic big box retailer translates social media chatter into Big Data to assess possibilities and probabilities for new products. An auto-parts retailer has learned how to segment its markets by vehicle distribution in given locales, allowing a customized (and higher-service, lower-cost) inventory profile in stores.

A major retail drug chain has watched as many cherished stocking principles and layout initiatives have gone by the boards, casualties of newfound, Big Data-driven insights into buying behavior by customer type and the surrounding retailing environment.

FOR THE FUTURE
It is interesting to consider and debate the value of fully integrated technologies spinning off real-time Big Data to avoid accidents, find optimal routes, and leverage transport capacity. Maybe some day ...

Greater minds than ours will find and test those advanced ideas. For now, we would be happy—even elated—to see real-world, practical results from supply chain planning and operations that draw from the learnings provided by smart use of Big Data. And we suspect that we are not missing all that much by skipping some of the over-hyped theoretic Big Data presentations at conferences populated by hungry consultants and credulous executives.