Databricks adds new developer tool to lakehouse platform

Databricks recently added a new tool to its lakehouse platform, enabling developers to author and test code in a familiar integrated development environment before connecting it to their Databricks cluster.

Founded in 2013 and based in San Francisco, Databricks is a data lakehouse vendor whose platform combines the structured data storage capabilities of data warehouses with the unstructured data capabilities of data lakes.

Although the vendor enabled developers work with their data in myriad ways, that work had to be done within the Databricks environment.

Now with the public preview of Visual Code Extension for Databricks, the lakehouse vendor is enabling developers to build data, augmented intelligence and machine learning models and applications with Visual Studio Code (VS Code) before moving it into Databricks.

VS Code is an integrated development environment (IDE) launched by Microsoft in 2015 that developers commonly use for a host of operations. That includes editing, testing, debugging and controlling continuous integration/continuous delivery (CI/CD) pipelines.

New capabilities

Visual Code Extension essentially enables developers to use a familiar tool in their most familiar environment before moving it into the Databricks lakehouse. That is significant for developers, according to Donald Farmer, founder and principal of TreeHive Strategy.

He noted that many developers find it frustrating to have to work outside their preferred development environment.

“[Visual Code Extension] is useful for developers who use Visual Studio,” he said. “It has been a real frustration for them to have to work in another IDE.”

Farmer added that when vendors force data workers to work in a particular environment rather than a familiar one, it can be a barrier to adoption. By launching Visual Code Extension, Databricks is attempting to make it easier for potential customers to work with the vendor’s lakehouse.

“This announcement shows that this has been a … barrier to adoption. So it’s good to see Databricks listening to the community and delivering,” Farmer said.

The community, in fact, played a substantial role in Databricks’ decision to develop Visual Code Extension, according to Tarek Madkour, the vendor’s director of product management.

As Databricks adds functionality to its lakehouse to both enable existing customers and attract new ones, its primary motivation is making data workers as successful as possible, Madkour said. In its attempt to do so, the vendor communicates frequently with customers to learn what users want added to the Databricks platform.

Familiarity is a common theme. Providing developers a familiar environment was motivation for developing Visual Code Extension.

“We want to meet developers where they are,” Madkour said. “We want to enable developers to use tools they are familiar with and make them productive with that. Visual Studio Code is one the [most popular] in the industry.”

Broader roadmap

Beyond extensions to IDEs, one part Databricks’ roadmap will focus on enhancing the data governance capabilities of its lakehouse platform, according to Madkour.

The vendor launched Unity Catalog in June 2022 after unveiling it in preview a year earlier, enabling organizations to more easily organize and govern their data. It plans to continue adding functionality.

In addition, Madkour noted that Databricks sees opportunities to develop integrations with new technologies like ChatGPT and other AI tools the vendor can use to develop no-code/low-code features.

“There’s a new world of generative AI — we’ve all heard about ChatGPT and other large language models. And that’s going to open up a whole new world of what we can enable our customers to do,” he said. “Think about people who don’t necessarily know code who would love access to data. That’s a new opportunity for Databricks.”

For the full article, please click below:

Databricks adds new developer tool to lakehouse platform | TechTarget

For the latest industry news, click here: https://searchdatagroup.co.uk/data-live-data-analytics-industry-news/