The Science of Decision Making Based on Big Data

Few weeks ago, we wrote a post about what’s next for all this big data being collected. We have a lot of data and the infrastructure to store it, but not using it to drive results is going to plunge businesses into big data disillusionment. It’s time to invest in a deeper understanding of analytics and what outcomes that data can drive.

Revisiting Gartner’s definition of big data reminds us that it is “high-volume, high-velocity and high-variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making.”

Enhanced insight and decision making: that’s what it’s all about right now. We call this decision science. At Indix, when we think about big data, we see three required elements. There’s the big data infrastructure that is required to store and manage the data. There’s the data itself, and the data science required to organize and structure the data. And finally there is decision science, which is the effort required to analyze and extract insights from the data.  It is these analytical processes that will ultimately help businesses drive desirable outcomes, whether it is revenue, profit, or loyalty.

In our last post, we highlighted some companies (Netflix, Knewton, Foursquare) that are already doing this – using decision science to drive desired outcomes. Decision science is the “art” of extracting insights that provide deeper understanding of a business’ ins and outs, and can actually move the needle when it comes to business goals.

I like to think of it as making a gourmet meal. It’s an art and science both. Raw ingredients on their own have potential but no utility. The ingredients have to be processed, sautéed, tempered, roasted, cooked to the right temperature, with the right tools and utensils, served with the right garnish, the right sauce, on the appropriate china. That’s what makes a perfect meal and delights the end user, whoever it may be. There is a learning process and science associated with putting the perfect creation on a plate. That’s kind of how it is with big data.

Data science is a combination of knowing what to look for, and then understanding how to get at it. It’s not just about collecting raw data. Decision science cannot be applied to raw and unstructured data. The data needs to be organized, normalized, and made ready for analysis. Then we need tools that can quickly and effectively analyze this data to provide a whole range of insights. This can be achieved through machine learning algorithms and other intelligent processes.

Moreover, it all needs to be easy enough for the broadest set of businesspeople to use. For instance, in a retail environment, a brand or category manager shouldn’t have to consult with the IT department every time s/he wants to makes sense of the data that is available to the business. That’s what makes decision science a really hard but fun and challenging problem to solve.

At Indix, the data science part of the activity includes collecting the data, organizing it in the appropriate databases, and then structuring the data to get it ready for decision science. Decision science for us includes analyzing the data, which involves creating lots of different algorithms to look for patterns in the data, then visualizing it for ready comprehension and, finally, personalizing the insights for the individuals in a business so that they can take action as it relates to their role in the organization.

Decision science emphasizes the need for domain expertise, just like in making a gourmet meal. Our domain expertise is in the area of products. We are building the world’s largest database of products to help businesses deliver the right product information to the right person at the right time, using the right channel. Our efforts in data science and decision science are informed by a deep understanding of product information and the business outcomes desired by companies that make products or sell products or deal with product information.

To implement decision science to its fullest capacity, we need to have a thorough understanding of the problems that need to be solved in that particular domain. For instance, if we think about Netflix, they want to ensure that people can easily find the entertainment to suit their mood and preference. At the end of the day, it’s thinking about the end user.

When it comes to product data science, we understand that businesses care about pricing, assortment, channels, product attributes and tags, MAP violations, known and unknown competitors, overall product ecosystem, market intelligence, and so on. By applying learning algorithms and patented technologies, we help businesses get access to these insights in a highly visual and user-friendly manner.

Every industry needs to do apply the same scientific approach with big data in order to make it work. It requires the right balance of art and scientific acumen to put the right ingredients together. What are you serving up today? The possibilities are endless.


  Download the Pervasive Commerce White Paper

Leave a Reply

Your email address will not be published. Required fields are marked *