An Analyst-Oriented Evolution of the Databricks Platform

What is Databricks SQL Analytics?

Today, Databricks announced the launch of SQL Analytics, which is a substantial evolution and another step towards realizing Databricks’ vision of the Lakehouse architecture. The new type of workspace, known as “SQL Analytics Workspace”, is available to Databricks customers and significantly enhances the experience of data analysts, data scientists, BI developers and business consumers alike, but while also offering substantial benefits to IT.

The new SQL Analytics Workspace exists independently of the existing Databricks Notebooks interface and resembles traditional SQL querying platforms – with all of the familiar functionality to explore tables and write queries as the well-established SQL syntax. SQL Analytics includes the ability to build visualizations and dashboards within the interface for quick analysis. Databricks has also partnered with Tableau and Power BI to ensure that connectors to these front-end BI & analytics tools offer simple, performant, and reliable access to data. Additional front-end tools including Qlik, Looker and ThoughtSpot, will support SQL Analytics connectivity in the near future. The SQL Analytics Workspace is a significant development for IT as it solidifies the Lakehouse vision and unifies architectures, tapping into the benefits of both Data Lake and Data Warehouse storage.

Why is this a big deal for Data Analysts and Consumers?

Databricks SQL Analytics brings the power of Databricks and Data Lakes to a much wider audience. Databricks facilitates data engineering and data science at scale, but to date, user comfort and capability hurdles have prevented more analysts and end-users from tapping into all that it has to offer. SQL Analytics encourages users to explore data in a more interactive, analyst-friendly workspace. They can leverage SQL as opposed to Python, R, or other languages, viewing query results as tables or visualizations. For analysts or data consumers who are less hands-on, insights can easily be surfaced via the most popular data visualization tools on the market today.

Thorogood and our customers have found Databricks to be unique in that it offers the gold-standard for analytics experimentation and ad-hoc modelling, but also enables operationalization and the deployment of scalable data science models for broad consumption. Visualizing, exploring, and understanding input data and model outputs, as well as testing and training results are critical for delivering data science applications that provide value – SQL Analytics provides a user-friendly interface with enhanced capabilities to enable this type of on-the-fly data exploration. This addition to the Databricks platform fosters greater collaboration between data science and data engineering, and ultimately streamlines data science project delivery by centralizing data exploration analysis and allowing users to spend more time analyzing results.

Why is this a big deal for IT?

The Lakehouse Vision Realized – A Paradigm Shift in Data Lakes and Warehousing

Data Warehouses held the preeminent position as the go-to, robust data repository to support enterprise reporting and analytics for many years. The advent of cloud computing ushered in the age of Data Lake storage. At a high level, Data Warehouses offer structure and ensure integrity of data, while Data Lakes enable storage of data of greater variety, velocity, and volume, at the cost of some structure and reliability. This costly trade-off is remedied by Databricks with its Delta Lake and SQL Analytics offerings – this model is colloquially referred to as the Lakehouse and offers the best of both worlds.

Databricks with Delta Lake offers Data Lake benefits such as cost efficiency and the ability to store larger volumes of more varied data entering systems at a higher frequency, but while also supporting key functionality, which is expected of a robust and structured data repository such as ACID transactions, schema enforcement, and data visualization connectors. SQL Analytics takes the Databricks Lakehouse proposition another step forward, as analysts and end users no longer have to face hurdles or unfamiliar terrain when accessing their data – they now have a familiar database-like querying interface and the ability to access data from their preferred data visualization tools.

A final point, but one that should not be understated, is that this is another stride forward in the enablement of multi-cloud strategies. Companies that load data into Data Lakes leveraging Delta Lake can now enable a broader user base to access that data in front-end tools or via SQL Analytics workspaces, and this functionality will be available uniformly across the Azure and AWS clouds. Therefore, the same Databricks code can be deployed on either cloud, minimizing dependencies on one or the other and reducing dependency on accessibility from certain storage technologies.

Learn More at Our Upcoming Co-Hosted Panel Discussion

Thorogood partners with Databricks and leverages Databricks for data engineering and data science use-cases across the AWS and Azure platforms to bring value to our customers. Thorogood has had hands-on experience trialing SQL Analytics while it was in private preview, so please feel free to reach out to Alaistair Jones with any questions you may have on Databricks, SQL Analytics, and the Lakehouse paradigm.

If you’re keen to learn more about this new offering or have any questions, we’re co-hosting an online panel and Q&A with Databricks on December 2nd which you can register directly for on our website here as well as a link at the very bottom of this page.