Any business that pursues a comprehensive analytics strategy will inevitably confront a frustrating paradox. Good decisions are built on a bedrock of maximum data.
Yet the more data you incorporate into your analyses, the more time you must spend preparing that data so that it can be analyzed. Look at any of today’s most progressive organizations and you’ll find that their first step toward a data-driven future was solving this conundrum.
In order to create an environment where data is consistently used to inform business strategy, a company must make it easy for that data to be used. More often than not, the solution is a centralized data mart that stores and standardizes all of a company’s data sources, thereby eliminating the time and expense of manual aggregation while allowing users to spend their time conducting the sorts of analyses that drive results. A recent Thorogood project for a prominent packaged goods manufacturer that had long struggled with the extra challenges of reconciling external data with internal data, highlights how seamlessly a company can realize these benefits by combining the right technology and the right development expertise.
Building a solid foundation for a data-driven future
The company in question had long wanted to equip executives in its head office with a series of dashboards that enabled them to quickly and intuitively monitor global performance. As the manufacturer of a specialized product that deals with intense marketplace competition, the company faces a variety of high-stakes decisions that depend upon an accurate understanding of consumer behavior. To achieve this understanding, the firm relies heavily on third-party vendors who provide datasets that track local markets. Analysts in each of the company’s 75 markets combine this data with internal figures to project demand, plan supply, and optimize price points.
For several years, our client had tried to come up with a solution that was capable of reconciling both internal and external data into a central repository to feed their reporting needs, but incorporating this information into an executive-level view of performance proved problematic.
Like many firms, our client knew that it would benefit from a centralized data warehouse where its internal and external data sources could be structured and stored in a uniform manner. But the company had some reservations about the potential disruption of a fundamental change to its architecture. They needed any systems migration to maintain a strict business focus that would not jeopardize the needs of end-users to accommodate the needs of IT. These concerns were well-founded, given the one-size-fits-all approach that internal IT departments often take because of the demands on their time and that too often, leave companies with a system that sacrifices local control in exchange for consistency and stability.
At the same time, our client needed uniformity. The dashboards it wanted to build relied on a mixture of external and internal data that was structured in a way that their architecture could not reconcile. The only solution was a great deal of manual collation, a process that would waste a great deal of time and increase the potential for faulty analysis due to data that had been compromised by human error. Our client knew that the technology existed to automate this process and create an environment where all of its data sources could be combined, standardized, and stored in a format capable of feeding reporting across a variety of platforms. They knew that such an environment would facilitate the implementation of the current reporting project and also encourage users to engage in future analyses. But it needed to develop the new environment in an elegant, business-focused manner that accounted for the company’s governance structure.
Minimizing disruption through specialized expertise
Thorogood’s combination of technological expertise and business acumen proved to be the answer. After consulting with company officials, our consultants proposed a solution that would build a bridge between the conflicting hierarchies of their external datasets and allow for their alignment with internal data from SAP. The system would be built around an Azure data mart that would make it easy to reach business end goals by harnessing the power and flexibility of the cloud. As a Gold Microsoft Partner and to minimize the time it would take to complete the project, Thorogood would host the solution on Thorogood’s CSP server, which would enable it to design and build the data mart without having to inconvenience the client’s IT department.
The approach to building the solution was a phased one: raw files would land and get stored in a Blob storage to then be cleansed and transformed using Databricks, and loaded into a back-end database using Azure Data Factory pipelines. On top of that database, consultants built an Azure Analysis Services cube, where users could then connect to the data from Excel, Power BI, or any other platform.
The end result was a solution that, with minimal disruption and at a reasonable cost, has provided one of the world’s largest consumer packaged goods firms with the right data, organized in the right way, that caters to the right use cases. Based off Thorogood’s expertise and ability to translate complex technical concepts into business speak, the company has become able to enjoy the benefits of the unified data repository it had longed for in close to no time. This has repositioned the company for future expansion of its analytics capabilities, being able to source insights from a powerful dataset of combined internal and external data and a golden source of product master data.
