Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.streamnative.io/llms.txt

Use this file to discover all available pages before exploring further.

This guide describes how to prepare a Databricks Unity Catalog for use with StreamNative Ursa as a managed Iceberg table catalog on Microsoft Azure.

​
Prerequisites

  • A Databricks workspace on Azure with Unity Catalog and Iceberg Managed Table enabled
  • An Azure subscription with permissions to create storage accounts and Access Connectors

​
1. Create an Azure Storage Container

Create a storage container in your Azure Storage Account (for example, unity-catalog-iceberg). The container path will follow the format: 
abfss://<container>@<storage-account>.dfs.core.windows.net
Create storage container Storage container

​
2. Create an Access Connector for Azure Databricks

Refer to the Azure Databricks Managed Identities documentation for the canonical procedure. In the Azure Portal, create an Access Connector for Azure Databricks. Access Connector Access Connector settings Access Connector created Record the connector Resource ID, which has the form: 
/subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.Databricks/accessConnectors/<connector-name>

​
3. Grant Storage Permissions to the Access Connector

The Access Connector identity requires the following roles:
ScopeRole
Storage AccountStorage Blob Data Contributor
Storage AccountStorage Queue Data Contributor
Resource GroupEventGrid EventSubscription Contributor

​
3.1 Grant Storage Blob Data Contributor

Grant Blob Data Contributor Grant Blob Data Contributor Grant Blob Data Contributor Grant Blob Data Contributor

​
3.2 Grant Storage Queue Data Contributor

Grant Queue Data Contributor Grant Queue Data Contributor Grant Queue Data Contributor

​
3.3 Grant EventGrid EventSubscription Contributor

Grant EventGrid Contributor Grant EventGrid Contributor Grant EventGrid Contributor Grant EventGrid Contributor

​
4. Create the Unity Catalog Metastore

Create the Unity Catalog metastore in Databricks. Create metastore Metastore configuration Metastore created

​
5. Create a Storage Credential

In the Databricks Catalog console, create a storage credential linked to the Access Connector created in step 2. Create credential Credential form

​
6. Create the External Location

Create an external location pointing to the Azure storage container:
  • URL: abfss://<container>@<storage-account>.dfs.core.windows.net
  • Storage credential: the credential created in step 5
Create external location External location settings

​
7. Create the Unity Catalog

Create a new Catalog and bind it to the external location created in step 6. Create catalog Catalog form

​
8. Grant Catalog Permissions

Grant permissions on the catalog. The EXTERNAL_USE_SCHEMA permission is required for Iceberg Managed Tables in Unity Catalog. Grant permissions EXTERNAL_USE_SCHEMA permission

​
9. Enable External Data Access on the Metastore

Note: This action requires Azure Account Admin privileges; without them, the Metastore entry is not visible.
Enable External data access on the metastore. This step is required for Unity Catalog Iceberg Managed Tables. Enable external data access External data access enabled

​
10. Create OAuth2 Credentials

Create an OAuth2 service principal that StreamNative Ursa will use to authenticate. OAuth2 setup OAuth2 setup OAuth2 setup Generate a secret for the principal and record both the Client ID and Client Secret. Generate secret

​
Catalog Information Summary

When the steps above are complete, collect the following values for the StreamNative Ursa compaction service:
ValueDescription
URIDatabricks workspace URL (e.g., https://adb-<workspace-id>.azuredatabricks.net/api/2.1/unity-catalog/iceberg-rest)
WarehouseThe Unity Catalog name created in step 7
Credential<client-id>:<client-secret> from step 10
For the next steps, see Configure Lakehouse Catalogs.