How Big Data Can Drive Competitive Intelligence

Companies increasingly mine their own customer data for insights into the market. But what about data that tracks your competitors’ activities in an ethical but profitable way?

Oh, sure. Coca-Cola isn’t likely to just send Pepsico a terabyte of sales data for kicks. But what if Pepsi knew how often people buy Coke at the supermarket, and what else is likely to be in their shopping cart at the same time?

It’s time for businesses to start a competitive intelligence process — while lowering the cost of analyzing information. In short, you need to continuously gather data online, from social media, website changes, news sources, and posted documents online, looking for clever bits of gold in the digital stream.

You’ll need to blend baseline of competitive intelligence (to prevent surprises in your own business) with proprietary data sets obtained on the market (to create surprises you’re your rivals).

Indeed, a universe of third-party data can add context to internal marketing data and can provide strategic insight into the vulnerabilities of competitors. Consider these three market trends:

1. Vendors like Acxiom, KBM Group, Bluekai and Datalogix have been increasingly vacuuming up that marketing data for resale. A study from the Tata Group looking at big data trends estimated that half of firms producing big data sets sell their digital data today. The average sale earned those companies $22 million in 2012.

2. Of the industries producing big data byproducts for sale, telecommunications firms and tech businesses tend to be the most prolific vendors as well as the most prolific users of external data, according to Tata. Insurance companies tend to make the most money from selling their data, however.

3. Manufacturing firms and energy companies tend to sell the least amount of big data, while consumer goods and media companies tend to use external data the least, despite the obvious value. That’s changing though, because it’s becoming easier to find buyers.

Third-party data is barely used, and it should be used more, said Keith Sayewitz, chief marketing officer and head of sales for the Big Data Exchange in Seattle, a Seattle startup that works like a stock market to trade Big Data sets. Imagine, if you’re a brick-and-mortar retailer, and every consumer walked in with a sign showing you what they had been shopping for in the last month.

The practice of selling data to the marketplace appears to be much more prevalent in Asia than in Europe or the United States, according to Tata. That may reflect regulatory considerations. U.S. data brokers generally ensure that big data sets have been stripped of individually-identifiable consumer information, both to ensure regulatory compliance and to prevent the inevitable public backlash. But it’s telling that China’s southwestern province of Guizhou is establishing an exchange,GBDex, to provide data cleaning, modeling, and data platform development. Alibaba is a partner in the exchange in Guiyang.

A small firm with a progressive attitude toward analytics may be able to carve out a competitive advantage against a much bigger rival simply by understanding their niche in the market better.

It’s data judo, using the weight of data of a larger rival to one’s own advantage if you’re not bogging down your marketing staff with lower-value tasks like vetting and cleaning information.

Gathering external data is rarely a core enterprise function for most firms looking at competitive intelligence. External data isn’t proprietary. Market research should center on the highest-value tasks data analysis and presentation of results.

As long as you have access to solid information both from internal and external sources the clever analysis of that data is what creates competitive advantages.