For the complete documentation index, see llms.txt. This page is also available as Markdown.

Connect to Databricks

Connecting Databricks to Finout gives your team visibility into Databricks spend across all your workspaces, alongside your other cloud and SaaS costs. A single account-level connection covers every workspace automatically, with costs broken down by workspace, cluster, SQL warehouse, job, and pipeline - no manual DBU rate configuration required.

With this integration you can:

  • Track Databricks spend alongside your other cloud and SaaS costs in a single view

  • Break down costs by workspace, cluster, SQL warehouse, job, and pipeline

  • Use Virtual Tags that include Databricks dimensions to allocate costs by team, project, or environment

  • Use Shared Cost Reallocation to distribute shared cluster or warehouse costs across teams

  • Include Databricks cost data in custom Dashboards, Anomalies, and financial plans

Finout uses read-only access to Databricks system tables and does not modify or delete any of your data. Queries consume Databricks compute credits (DBUs).


Prerequisites

Before connecting, make sure you have:

  • A Databricks account with account admin privileges

  • Unity Catalog enabled on any workspace — see the Databricks documentation for AWS, Azure, or GCP

  • Metastore owner status for the Unity Catalog-enabled workspace


1. Set up in Databricks

Step 1 — Collect your Account ID and Workspace URL

  1. Log in to the Databricks account console.

  2. Click your avatar in the top right and copy your Databricks Account ID — save this for the steps below.

  3. Click Workspaces and select a Unity Catalog-enabled workspace.

  4. Copy your Workspace URL — save this for the steps below. Then open the workspace.


Step 2 — Create a service principal

  1. Click your avatar in the top right of the workspace and select Settings.

  2. Under Workspace admin, select Identity and access.

  3. Next to Service principals, click Manage.

  4. Click Add service principal > Add new.

  5. For Service principal name, enter finout-billing-serviceprincipal, then click Add.

  6. Open the newly created service principal and select the Secrets tab.

  7. Click Generate secret.

  8. Set a Lifetime of 730 days, then click Generate.

  1. Copy both the Client Secret and Client ID — save these for the Finout connection.


Step 3 — Create a SQL warehouse

  1. From the left navigation, under SQL, click SQL Warehouses.

  2. Click Create SQL warehouse and enter the following:

Field
Value

Name

finout-billing-warehouse

Cluster size

2X-Small

Type

Serverless

  1. Click Create.

  2. In the Manage permissions modal which opens next, search for and select finout-billing-serviceprincipal.

  3. Set the permission to Can Use and click Add.

  4. Close the modal and copy the Warehouse ID displayed next to the warehouse name — save this for the connection with Finout.


Step 4 — Grant data reader permissions

  1. From the top of the left navigation, click Catalog.

  2. Expand My Organization > system.

  3. For each of the following schemas, select the schema, click Permissions > Grant, add finout-billing-serviceprincipal as the principal, set Privilege presets to Data Reader, and click Confirm:

    • system.access

    • system.billing

    • system.compute


2. Connect Databricks to Finout

  1. In Finout, navigate to Settings > Cost Centers.

  2. Click Add Cost Center and select Databricks. The Connect Databricks wizard opens.

  1. Enter the following details:

Field
Description

Cost Center Name

A name for this Cost Center in Finout

Service Principal Client ID

Client ID of the service principal created in Databricks

Service Principal Client Secret

Client secret generated in Databricks

Account Workspace URL

Workspace URL collected in Databricks (e.g. acme-inc.cloud.databricks.com) Make sure to remove your URL schema (e.g. https://) and any query params or paths following the top level domain (any ?xxx or /xxx after .com, .net etc.)

  1. Click Next. Finout validates your credentials, warehouse access, and permissions on all required system tables. If validation fails, an error message identifies the specific issue to fix.

Result: Your Databricks Cost Center is created, and data should be available within 24 hours.


Cloud infrastructure costs

Finout reports Databricks DBU usage costs. These do not include the underlying cloud infrastructure costs (compute, storage, networking) that Databricks runs on - those appear separately in your cloud provider cost center (AWS, GCP or Azure).

To See Databricks Cloud Costs Alongside Native Databricks costs

You can use a Virtual Tag to see Databricks-attributable cloud infrastructure spend alongside your DBU costs, without merging or double-counting the underlying dollar amounts

Group the relevant cloud provider costs (e.g. by resource tag or naming convention tied to Databricks) into the same view as your Databricks Cost Center.


Still need help? Reach out to our team at support@finout.io.

Last updated

Was this helpful?