Databricks introduces an open-source tool for sharing data

Databricks, an industry-leading, cloud-based data engineering tool, has launched its new open-source project called Delta Sharing. It is designed to be a vendor-neutral way to share data with any cloud infrastructure or SaaS product. It is also part of the broader Databricks open-source Delta Lake project.

Databricks is commonly used for processing and transforming massive quantities of data and exploring the data through machine learning models. “The number one barrier for organizations to succeed with data is sharing data, sharing it between different views, sharing it across organizations — that’s the number one issue we’ve seen in organizations,” CEO Ali Ghodsi explained.

Delta Sharing is an open-source protocol designed to solve the data sharing problem. This is the industry’s first-ever open protocol, an open standard for sharing a data set securely. The tool is designed to work with multiple cloud infrastructure and SaaS services, and out of the gate, there are multiple partners involved, including the infrastructure vendors like Amazon, Microsoft, and Google, as well as data visualization and management vendors like Qlik, Starburst, Collibra and Alation and data providers like Nasdaq, S&P and Foursquare.

Ghodsi said the key to making this work is the open nature of the project, as he believes in keeping the service open to other vendors. He also donated to The Linux Foundation to ensure product availability across different environments. There are a large number of connectors available today, but tying up with popular cloud platforms will ensure even more connections to the services.