With Databricks Apps, business users get more out of data

The Databricks platform now offers new opportunities to put data into the hands of business users. For them, it is useful to do more with insights from data, but that usually involves a lot of data engineering work. Databricks Apps should change that by giving developers additional options for building applications.

Business users can make informed decisions quickly when they have those high-quality insights. Something like that can be useful for sales and marketing staff. Think of a marketing department creating dashboards to visualize campaign performance. Suddenly, they can easily and quickly extract insights from data themselves. A bonus: they can immediately use their knowledge of marketing in general and the specific situation directly to interpret the data.

Databricks Apps has multiple use cases. For example, the new product can be used to build applications that rely on machine learning models for predictive maintenance or fraud detection. The new tool can also be used to create apps to track and improve data quality.

From challenge to solution

On that note, making data more widely available within the organization is a welcome addition. Many data platforms have also achieved this in recent years, but there is still room for simplification. In practice, the ambition to bring insights to business users still too often means that engineers perform work around data quality and data retrieval using SQL. A lot of time goes into setting up and managing infrastructure and implementing access and governance mechanisms.

At Databricks, they set out to solve these challenges. It developed Databricks Apps. Within the Databricks Data Intelligence Platform, a user can quickly click on “create” and then “app,” after which a template can be selected. After this, you can choose whether to work with the Python frameworks Dash (for chatbots), Gradio (for chatbots or apps), or Streamlit (for chatbots or apps). If you want to create more customized applications, the frameworks Shiny and Flask can be used.

The built apps run in the Data Intelligence Platform, providing direct access to data and AI models. In this way, Databricks Apps is useful, for example, for actually converting a RAG proof-of-concept into an application. Several companies have made efforts to develop RAG capabilities, but implementation may still lag behind. For example, such a RAG system can be more easily deployed to answer user questions, relying on proprietary knowledge residing in the Data Intelligence Platform.

Tip: After the AI world, Databricks now wants to change the analytics market

Ready for production

As we described above, building infrastructure traditionally requires a lot of work. Databricks has solved this by running apps on automatically provisioned serverless compute, which allows the process to move fairly quickly. Within the Data Intelligence Platform, users click a “Deploy” button, and the app is actively running within moments.

In addition, Databricks Apps can also work with the Visual Studio Code and PyCharm development environments. These two environments (IDEs) are popular within the data science community. Therefore, Databricks Apps integrates with existing developer workflows. It does not matter whether the Databricks workspace is used or if it is a third-party IDE: there is support for Git version control and CI/CD pipelines.

Data security

Implementing security mechanisms is an automatic priority when developing data apps. Therefore, Databricks Apps has immediately taken care of some basic things for that. In principle, the data remains within the Databricks environment. Only when explicitly chosen to share data leaves the environment.

Why is this necessary? When data stays within the Data Intelligence Platform, it automatically uses protection mechanisms. These include granular access controls for precise data permissions, automatically managed service principals for secure communication between applications, automatic user authentication via OIDC/OAuth 2.0, and single sign-on for secure access.

In addition, organizations can leverage Databricks’ Unity Catalog for lineage options. This gives them insight into application data origins, data transformation, and usage. Tracing changes has the advantage of allowing you to stay on top of following business policies and general laws and regulations.

Databricks Apps is available immediately as a public preview on AWS and Microsoft Azure.