Working With Product Data on the Internet: Part 1 - Indix



Working With Product Data on the Internet: Part 1

This is the first in a series of posts about the challenges of working with product data on the internet.

Retail and commerce are undergoing a major transformation today, and the internet is fundamentally changing the landscape. Consumers all over the world are embracing and fueling this transformation as they find the convenience and ubiquity of this convenience too alluring to pass. As more and more commerce moves to the internet, the typical actors in the retail and commerce ecosystem need to gear up to succeed in this new digital front. Whether it’s a brand, retailer, commerce hub, or retail analytics firm, businesses now have the additional task of collecting, processing, and organizing data from across the internet to make the most informed business decisions.

While the internet offers little resistance to those who wish to mine the web for nuggets of information, its scale and non-standardized nature make it extremely difficult to smelt these nuggets into pure gold.

Today, we will delve deeper into this problem space and surface the top challenges in this domain. Subsequent posts will focus on strategies and tools that can help navigate this space better and with increased confidence.

Defining the “Who” and “What” of the Problem Space

We briefly touched upon the key actors in the commerce space above. Now let’s take that one level deeper and define the top use cases for them.


On the surface, the actors and their intents look so different that we would almost assume that each has their own unique challenges. In reality though, these differences are only skin deep. Below the epidermis, the challenges do not look so different, and we can group them across a few root causes.


We knowingly left out what would have been “Challenge 0” for each of the rows: Scale – more precisely, the challenge of getting product data at scale. For example, let’s take the use case of a retailer trying to analyze its assortment against competitors. This would be easy to do against say one or two competitors, but as this number grows, any brute force manual or semi-automated process that may have worked at a lower scale looks extremely impractical. This challenge of scale applies equally well to any of the other use cases: it is one thing to track MAP violations of sellers on one store, but as the number of sellers grow, the challenge posed by scale is very evident.

Coming Up

In the next post of this series, we will take up a few real life examples that will illustrate the challenges discussed here more deeply. For those among us who are eager to jump directly to the last edition of this series where we look at some of the strategies and tools to navigate this problem better, please check out the recording of our recent webinar on this topic.

Also published on Medium.

  Download the Pervasive Commerce White Paper
Webinar - The Internet Is a Dirty Place

Leave a Reply

Your email address will not be published. Required fields are marked *