Azure SQL Data Warehouse - Cloud Data Warehouse, Data Virtualization and Real-Time all in one

Blog by Andrew Kennedy10 Oct 19

Microsoft Azure SQL Data Warehouse is an impressive cloud data warehouse in its own right, but its true value is best realized when combined with the other tools and services available on the Azure platform. Here I’ll talk through three different usage patterns that show how Azure SQL DW delivers valuable insights to end-users.

My first introduction to Azure SQL DW was when Thorogood helped Microsoft demonstrate to a large global consumer goods manufacturer just how effective it is in replacing large-scale on-premise analytics warehousing solutions. Having made an early foray into the cloud, figuring out when to scale the platform up and down was something that took a little getting used to, but improving the performance while reducing the total cost of ownership was a real winner.

Since those early days, Microsoft has continued to improve Azure SQL DW at pace, adding features like automatic statistics creation for easier maintenance, enhanced loading using Azure Data Factory, and data governance options like row- and column-level security. Tasks like implementing the right indexes are no less vital to that all-important performance, but Azure SQL DW now has tools to help identify the most common queries and optimize performance based on usage.

USE CASES

Azure SQL DW is an option you should consider across a range of scenarios:

Cloud Data Warehouse

SQL DW offers a reliable, performant, large-scale data warehousing platform that supports storing and analyzing petabytes of data. The flexible scaling options allow scaling when you need it with a recent study by Gigaom ranking it most cost-effective for performance against competitors like Amazon Redshift, Google BigQuery and Snowflake.

For end-users, it’s the combination of data and superior performance from SQL DW and the flexible data modeling and user-friendly visalization capabilities in Power BI that open up new opportunities to explore the data and find new insights that drive better decisions in their day-to-day work. Adding in AI capabilities using tools like Cognitive Services to provide sentiment analysis or extract keywords from text takes analytics to the next level, unlocking insights from previously untapped data such as user comments or customer feedback.

Applying structure to your Data Lake with Data Virtualization

In parallel to the need for a scalable, performant Data Warehouse, most organizations have large, varied collections of data that is collected and stored, but isn’t often processed. With the built-in PolyBase tool, it’s possible to store ever-increasing volumes of data in your Data Lake using familiar SQL queries.

One key target in the Data Lake philosophy is to make more data available for easier analysis – it’s a major win when we make data available in a way that offers structure that gets users to the analysis stage quicker in tools such as Excel and Power BI.

Real-time data analysis

This is a use-case that really takes advantage of the range of tools available in the Azure platform to open up some exciting analysis scenarios. Real-time examples I’ve seen with customers include monitoring machines in a factory, tracking users of a website and analyzing social media data.

We could use Azure Event Hub to receive data from Twitter, for example, and then Azure Databricks to run analysis on the data in real-time, and finally Azure Databricks’ enhanced connector to Azure SQL DW to store data, so that the latest Tweets are available in the database. Taking it a step further, we could have Databricks trigger an alert if the customer sentiment detected in the Tweets hits a particularly high point, and send an email to the marketing team suggesting they look into how we can capitalize on the attention.

CONCLUSION

Azure SQL Data Warehouse is an impressive offering from Microsoft, and as you can see it only gets better when combined with the rest of the Azure platform to give users access to data and analysis in a way that’s easier and more powerful than ever before.

If you’d like to take more of a look at SQL Data Warehouse, reach out to me on LinkedIn or at andrew.kennedy@thorogood.com or join Thorogood for one of our events which will give you a chance to see SQL DW in action.

Find out more

Contact Andrew Kennedy. Andrew is a BI & Analytics Consultant at Thorogood.