How data harmonization can help CPG companies win both online and in-store

Retail eCommerce is expected to rise eight per cent globally through 2024, with food and beverage categories expected to experience among the highest growth rates through 2026.¹

8 per cent of consumers are now shopping for groceries both in-store and online. This has placed an even greater premium on data, and the need to understand both online and in-store markets and performance both separately and together.²

Challenges in obtaining and planning for a total market view

Many CPGs and retailers still have separate in-store and eCommerce teams and departments, and often these teams use different datasets. eCommerce performance data may reside in one place, in-store performance data in another, and shipment performance data in yet another.

This can make holistic omnichannel sales forecasting and brand planning difficult, with no way to analyze across channels to understand common trends and problems.

Syndicated brick-and-mortar datasets need to be paired with eCommerce datasets to better understand the shifts occurring between the two channels. A cohesive data set is required to get a complete view of business performance.

Enter data harmonization.

What is data harmonization?

Data harmonization brings together disparate and various data sources, verticals, file formats, and naming conventions into a single database and then reorganizes it all into a single schema. The data is gathered and visible in a single, comprehensive, true view of business performance. It is customizable by an organization’s business model, and views can be automated and made simple with the creation of dashboards.

Data harmonization is a continuous task that keeps past, present and future data easy to understand so analysis can be regularly conducted. This includes continuous validation of data points across channels to ensure metric comparisons are accurate. It also means new attributes are created to assist with analysis as datasets and information are added.

The ultimate goal of data harmonization is to make business data “analytics ready.”

Why data harmonization is important for eCommerce

When executed properly, data harmonization results in clear insights that help CPG companies win both online and on the shelf. It provides an all-outlet total market view so a brand’s total retail sales are visible, along with their share of eCommerce and share of physical store sales. This enables a view of key performance indicators (KPIs) by channel, brand and segment.

Data harmonization assists with more accurate reporting by creating consistent values for the same metrics. It saves time through a central database, rather than having to collect information manually from several different departments. The central database also helps avoid teams duplicating the same tasks. It can lower the costs required to purchase and clean outside data. And it aids the storage of historic data such as for rebrands and consolidations. It creates confidence in data used for decision making across the organization.

How data harmonization works

The entire data harmonization setup process can be completed in as little as a few months, or in just a few days for adding new sources.

Here are the key steps:

1. Acquiring the data from multiple sources. Information input types may include market (syndicated), consumer (research and loyalty), trade and revenue trends, account (shipments, POS sales, store level information), trade/RGM (baseline, promotion), product (master data, pricing), external (such as weather) and other e.g. currency rates.

2. Ingesting the data into a combined, cleansed, centralized source.

3. Harmonizing and customizing the data based on the business and product hierarchy so that multiple data sources will have a consistent description viewable in one place. The harmonization process includes extraction (reading the data in the original data set), converting its format, and writing it into a new central database. The information is then blended together using machine learning and a ‘rules engine’ applied using product/item information, markets and geographies, dates and time periods, and units of measure.

4. Enriching the data with custom market metrics such as productivity that indexes an item’s velocity performance to the segment or category.

5. Analyzing the data to gain actionable insights by channel, brand, and customized subsegments. Outputs can be made to the data harmonization supplier’s analytics tools, or to the company’s business intelligence environment.

With a harmonized data set for analysis, understanding the growth between online and offline channels and the drivers behind them is possible to drive better business strategy and decision making, using a view of the holistic picture.