Mining Data in the Mining Sector: A Case Study in the Power of Unified Data Sources

Case study by James Leonard

The analytics revolution has created a world in which companies are relying almost as much on external data as their own internal figures. Across all industries, firms are finding themselves increasingly dependent on data from a variety of different vendors.

But maximizing the business impact of that data requires the unification of these disparate sources into a structured, coherent whole, as well as a solution that is robust enough to allow for flexible access to this data. Accomplishing this requires a thoughtfully designed platform specifically tailored to a company’s data and engineered to yield the business insights it seeks.

A recent Thorogood client is one example of a company whose business operations demand clean, coherent, standardized data that has been optimally mapped and prepared for the task at hand. A multinational mining corporation, the company relies heavily on a wide array of internal and external data sources to inform its analysis of market factors, a requirement that led the client to enlist Thorogood to reimagine the way in which it utilizes this data.

Making data digestible

The development of any business intelligence solution requires both a thorough understanding of technology and a thorough understanding of how that technology can best be applied to the specific business problems a company faces. Before Thorogood could develop a blueprint for the project, it needed to understand the needs and pain points of the users who would be utilizing the system.

The challenge that the client faced was a common one. Firms in the mining sector rely upon a wide array of external and internal data sources to inform their analysis of market factors. The greater the supply of data, the greater the challenge in gleaning the signal from the noise. In order to accurately project commodity prices, it needed to identify the most relevant performance indicators within its trove of market, commodity, and macroeconomic data sources. And it needed to give its analysts access to this data in a targeted, comprehensive and easily-digestible format so that they could react to an ever-changing market in as close to real-time as possible.

To that end, Thorogood consultants devised a web application that utilized AWS technology and Microsoft’s Power BI to provide the firm’s Market Analysis team with a holistic view of the factors driving global commodity prices and would allow users to quickly analyze a myriad of external data sets. The goal was to give users quick access to the key performance indicators they relied upon most while also enhancing their ability to actively analyze data on a case-by-case basis.

Built using the Python-based Django web framework, the Django REST API, and React; the Thorogood-developed app allowed users to select their desired configurations of prices, geographic locations, time intervals, and currencies, among others, by connecting to a single data layer which contained all of the company’s consolidated data. This data layer would pull raw data from various external sources, standardize it, and feed a semantic data model of SQL Server databases hosted in EC2 instances on the AWS cloud that would be aligned to corporate master data, ensuring consistency. Thorogood could then use this model to serve the needs of Power BI, Excel, and browser users for quick access to the key performance indicators they relied upon most.

In building the front-end Power BI dashboards, Thorogood worked side-by-side with business users to create a user interface that distilled the most relevant KPIs into the most user-friendly format. Consultants built the dashboards using Thorogood’s Iterative Dashboard Methodology, which allowed users to offer feedback until the dashboards met their needs. The final result was a platform that automated much of the analysis that users had previously been performing manually, while at the same time giving them the freedom to experiment with the data and perform their own freelance analyses. The app not only allows to make selections on commodities and time periods and dump those selections into Power BI for analysis, but it also allows users to export the selected data to CSV or Excel formats, extending the flexibility of using other tools to analyze it.

Ensuring access and adoption

The Thorogood-built app is changing the way the client does business. Being fed by a robust, unified data layer, users can now focus their energies on the analyses that matter most instead of spending time loading disparate datasets into Power BI and analyzing them in isolation. The solution is an excellent example of the benefits a company can realize by combining its various data sources into a unified whole while thinking holistically about business needs. With the new platform in place, the company’s Market Analysis team now has a powerful tool to fulfill its mission-critical analyses.

Find out more

Contact James Leonard for more information. James is a Data & AI Consultant at Thorogood and UK Managing Director