Documentation Index
Fetch the complete documentation index at: https://docs.streamnative.io/llms.txt
Use this file to discover all available pages before exploring further.
Preparing an external catalog is optional. StreamNative Ursa supports the following catalog modes:
- Iceberg with the Hadoop catalog (no external catalog). Iceberg metadata and data files are written directly to the configured object storage path. No external catalog service is required, and downstream engines can read the table by pointing at the storage location.
- Delta Lake without a catalog. Delta tables are written directly to the configured storage path; readers point at the path (
s3://..., gs://..., or abfss://...) to query them.
- External catalog. Use a managed catalog service (Databricks Unity Catalog, Snowflake Open Catalog, Snowflake Horizon Catalog, AWS S3Table, or Google BigLake) to register the tables. This is required if you want governance, discoverability, or integration with managed query engines (Databricks SQL, Snowflake, Athena, BigQuery, etc.).
If you choose to use an external catalog, this page is the index for the catalog preparation guides. Each guide focuses solely on provisioning the catalog itself and its supporting cloud resources (storage bucket, IAM role or service principal, and credentials). Once the catalog is ready, see Configure Lakehouse Catalogs for how to wire it into the compaction service.
Skip this page if you are using the Hadoop catalog (Iceberg) or no catalog (Delta). Go directly to Configure Lakehouse Catalogs, which describes the no-catalog configuration.
Catalog Support Matrix
| Catalog | Table Format | AWS | GCP | Azure |
|---|
| Databricks Unity Catalog (Managed Iceberg) | Iceberg | AWS | GCP | Azure |
| Databricks Unity Catalog (Delta Lake) | Delta Lake | AWS | GCP | Azure |
| Snowflake Open Catalog (Polaris) | Iceberg | AWS | GCP | Azure |
| Snowflake Horizon Catalog | Iceberg | AWS | — | — |
| AWS S3Table | Iceberg | Iceberg | — | — |
| Google BigLake | Iceberg | — | Iceberg | — |
Choosing a Catalog
| Use case | Recommended catalog |
|---|
| Manage Iceberg tables alongside Databricks workloads | Unity Catalog (Iceberg) |
| Manage Delta Lake tables alongside Databricks workloads | Unity Catalog (Delta Lake) |
| Cloud-portable Iceberg REST catalog from Snowflake | Snowflake Open Catalog (Polaris) |
| Governed Iceberg tables managed by Snowflake Horizon | Snowflake Horizon Catalog |
| AWS-native Iceberg tables with built-in Athena/Redshift integration | AWS S3Table |
| GCP-native Iceberg with BigQuery/BigLake integration | Google BigLake |
Catalog Preparation Guides
Databricks Unity Catalog (Managed Iceberg Table)
Use Unity Catalog to manage Iceberg tables governed by Databricks. The compaction service writes to Iceberg Managed Tables; downstream readers query them via the Unity Catalog Iceberg REST endpoint.
Databricks Unity Catalog (Delta Lake)
Use Unity Catalog to manage Delta Lake tables. The compaction service writes Delta Lake files governed by Unity Catalog and queryable from Databricks SQL or external Spark sessions.
Snowflake Open Catalog (Polaris)
Snowflake Open Catalog (Polaris) is a cloud-agnostic Iceberg REST catalog operated by Snowflake. It can be used as the catalog for Iceberg tables hosted on AWS, GCP, or Azure object storage.
Snowflake Horizon Catalog
Snowflake Horizon provides governed Iceberg tables with native Snowflake integration. The catalog uses a Snowflake External Volume backed by an S3 bucket and authenticates via a Programmatic Access Token (PAT).
AWS S3Table
AWS S3Table is the AWS-native Iceberg catalog with first-class integration into AWS analytics services such as Athena, Redshift, and EMR.
Important: The Ursa cluster must run in the same region as the S3Table bucket. Cross-region access is not supported.
Google BigLake
Google BigLake provides an Iceberg REST catalog tightly integrated with BigQuery and Google Cloud Storage.
Important: The Ursa cluster, GCS bucket, and BigLake catalog must all be in the same region. Each BigLake catalog maps to exactly one GCS bucket (1:1 mapping; sub-paths are not supported).
Next Steps
After preparing your catalog, proceed to:
- Dynamic Configuration Guide — Reference for all dynamic configuration keys and the cluster-name prefix requirement.
- Configure Lakehouse Catalogs — Connect the prepared catalog to the StreamNative Ursa compaction service.
- Enable Lakehouse Integration — Enable SDT (External Table) at the cluster, namespace, or topic level.
