> ## Documentation Index
> Fetch the complete documentation index at: https://docs.streamnative.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Deploy Lakehouse Table

This guide walks through the Kubernetes resources needed to deploy Lakehouse Table in a StreamNative Private Cloud environment.

## Prerequisites

* A StreamNative Private Cloud environment with the StreamNative operator (`PulsarCoordinator`) installed
* An object storage bucket (AWS S3, GCS, or Azure Blob Storage)
* An IAM role or service account with read/write permissions on the bucket
* Kubernetes namespace (e.g., `pulsar`) for deploying resources

## 1. ServiceAccount and IAM

Create a Kubernetes ServiceAccount that binds to your cloud IAM role. This grants the Pulsar broker and compaction service access to the object storage bucket.

### AWS (EKS)

```yaml theme={null}
apiVersion: v1
kind: ServiceAccount
metadata:
  name: private-cloud-broker
  namespace: pulsar
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::<your-account-id>:role/<your-role-name>
```

The IAM role must have the following permissions on the storage bucket:

```json theme={null}
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject",
        "s3:ListBucket",
        "s3:GetBucketLocation"
      ],
      "Resource": [
        "arn:aws:s3:::<your-bucket>/*",
        "arn:aws:s3:::<your-bucket>"
      ]
    }
  ]
}
```

### GCS (GKE)

Coming Soon.

### Azure (AKS)

Coming Soon.

## 2. Secrets

Create secrets for broker credentials and cloud provider configuration:

```yaml theme={null}
apiVersion: v1
kind: Secret
metadata:
  name: pulsarcluster-private-cloud
  namespace: pulsar
type: Opaque
stringData:
  broker_client_credential.json: "{}"
---
apiVersion: v1
kind: Secret
metadata:
  name: pulsarcluster-private-cloud-aws
  namespace: pulsar
type: Opaque
stringData:
  config: |
    [default]
    region = <your-region>
```

## 3. PulsarCoordinator

Deploy the StreamNative operator:

```yaml theme={null}
apiVersion: k8s.streamnative.io/v1alpha1
kind: PulsarCoordinator
metadata:
  name: private-cloud
  namespace: pulsar
spec:
  image: streamnative/private-cloud:4.2.0.5
```

## 4. Oxia Cluster and Namespaces

Oxia provides metadata storage, leader election for the compaction service, and offset index management.

### OxiaCluster

```yaml theme={null}
apiVersion: k8s.streamnative.io/v1alpha1
kind: OxiaCluster
metadata:
  labels:
    k8s.streamnative.io/coordinator-name: private-cloud
  name: private-cloud
  namespace: pulsar
spec:
  monitoringEnabled: true
  image: oxia/oxia:0.16.2
  imagePullPolicy: IfNotPresent
  server:
    replicas: 3
    persistentVolumeClaimRetentionPolicy:
      whenDeleted: Delete
```

### OxiaNamespaces

Three namespaces are required:

```yaml theme={null}
# Broker metadata (3 shards, replication factor 3)
apiVersion: k8s.streamnative.io/v1alpha1
kind: OxiaNamespace
metadata:
  name: broker
  namespace: pulsar
spec:
  namespaceConfig:
    name: broker
    initialShardCount: 3
    replicationFactor: 3
  clusterRef:
    name: private-cloud
    namespace: pulsar
---
# Schema storage (1 shard, replication factor 3)
apiVersion: k8s.streamnative.io/v1alpha1
kind: OxiaNamespace
metadata:
  name: ursa-schema
  namespace: pulsar
spec:
  namespaceConfig:
    name: ursa-schema
    initialShardCount: 1
    replicationFactor: 3
  clusterRef:
    name: private-cloud
    namespace: pulsar
---
# WAL storage metadata (32 shards, replication factor 3)
apiVersion: k8s.streamnative.io/v1alpha1
kind: OxiaNamespace
metadata:
  name: ursa-storage
  namespace: pulsar
spec:
  namespaceConfig:
    name: ursa-storage
    initialShardCount: 32
    replicationFactor: 3
  clusterRef:
    name: private-cloud
    namespace: pulsar
```

| Namespace | Shards | Purpose |
| - | - | - |
| `broker` | 3 | Broker metadata and coordination |
| `ursa-schema` | 1 | Schema version storage |
| `ursa-storage` | 32 | WAL storage metadata and offset index |

## 5. StorageCatalog

The StorageCatalog resource connects the Pulsar broker to Oxia and object storage:

```yaml theme={null}
apiVersion: k8s.streamnative.io/v1alpha1
kind: StorageCatalog
metadata:
  name: private-cloud
  namespace: pulsar
spec:
  oxiaMetadataServiceUrl: oxia://private-cloud-oxia:6648/broker
  storageUrl: oxia://private-cloud-oxia:6648/ursa-storage
  schemaStorageUrl: oxia://private-cloud-oxia:6648/ursa-schema
  backendStorageType: S3
  bucket: <your-bucket-name>
  region: <your-region>
  prefix: private-cloud/storage
  useOwnStorage: true
```

| Field | Description |
| - | - |
| `oxiaMetadataServiceUrl` | Oxia endpoint for broker metadata |
| `storageUrl` | Oxia endpoint for WAL storage metadata |
| `schemaStorageUrl` | Oxia endpoint for schema storage |
| `backendStorageType` | `S3`, `GCS`, or `AZUREBLOB` |
| `bucket` | Object storage bucket name |
| `region` | Bucket region |
| `prefix` | Key prefix within the bucket |
| `useOwnStorage` | `true` to use the configured bucket for WAL storage |

## 6. PulsarBroker with Compaction Service

The PulsarBroker resource includes the compaction scheduler configuration that enables lakehouse integration:

```yaml theme={null}
apiVersion: pulsar.streamnative.io/v1alpha1
kind: PulsarBroker
metadata:
  name: private-cloud
  namespace: pulsar
  annotations:
    cloud.streamnative.io/config-profile: default-config-v2
    cloud.streamnative.io/open-telemetry-enabled: "true"
    cloud.streamnative.io/open-telemetry-exporter: "prometheus"
    cloud.streamnative.io/open-telemetry-exporter-prometheus-port: "9464"
  labels:
    k8s.streamnative.io/coordinator-name: private-cloud
spec:
  image: streamnative/private-cloud:4.2.0.7
  replicas: 1
  zkServers: private-cloud-zk:2181
  config:
    clusterName: private-cloud
    useStorageCatalog: true
    managedLedgerOffloadConfig:
      enabled: true
    compactionScheduler:
      enabled: true
      replicas: 1
      pod:
        jvmOptions:
          extraOptions:
            - -Dotel.sdk.disabled=false
            - -Dotel.java.disabled.resource.providers=io.opentelemetry.instrumentation.resources.ProcessResourceProvider
            - -Dotel.metrics.exporter=prometheus
            - -Dotel.exporter.prometheus.port=9464
      config:
        deployMode: StatefulSet
        backendStorageType: S3
        cloudStorageConfig:
          bucketName: <your-bucket-name>
          region: <your-region>
          prefix: private-cloud/compaction
        lakehouseType: iceberg
        streamTableMode: EXTERNAL
    function:
      enabled: false
      mesh:
        uploadEnabled: false
  pod:
    serviceAccountName: private-cloud-broker
    resources:
      requests:
        cpu: 200m
        memory: 512Mi
    securityContext:
      runAsNonRoot: true
```

### Key Configuration Fields

| Field | Description |
| - | - |
| `useStorageCatalog: true` | Enables Ursa storage on the broker |
| `managedLedgerOffloadConfig.enabled: true` | Enables tiered storage offloading |
| `compactionScheduler.enabled: true` | Enables the compaction service |
| `compactionScheduler.replicas` | Number of compaction service pods |
| `config.backendStorageType` | Storage backend: `S3`, `GCS`, or `AZUREBLOB` |
| `config.cloudStorageConfig.bucketName` | Bucket for compacted lakehouse data |
| `config.cloudStorageConfig.region` | Bucket region |
| `config.cloudStorageConfig.prefix` | Key prefix for compacted data |
| `config.lakehouseType` | Table format: `iceberg`, `delta`, or `delta_and_iceberg` |

### Compaction Service Tuning (Optional)

Additional properties can be added under `compactionScheduler.config`:

| Property | Description | Default |
| - | - | - |
| `compactedFileSizeLimit` | Maximum Parquet file size before flush | 256 MB |
| `tailCompactDataVisibilityIntervalInSeconds` | Delay before data becomes visible in lakehouse | 180s |
| `maxCommitIntervalInSeconds` | Maximum interval between commits | 180s |
| `walReadRateLimitInBytesPerSecond` | WAL read throughput rate limit | 50 MB/s |
| `lakehouseCommitMaxRetryTimes` | Maximum retries for failed commits | 3 |
| `compactedThreadNum` | Number of compaction worker threads | CPU count - 1 |
| `commitThreadNum` | Number of commit threads | CPU count |

## 7. Cloud Provider Variants

### GCS

Coming Soon.

### Azure

Coming Soon.

## Next Steps

After deploying the infrastructure, proceed to:

1. [Prepare Lakehouse Catalogs](/private-cloud/v2/configure-private-cloud/private-preview/ursa-lakehouse/prepare-lakehouse-catalogs) -- Set up your external catalog service
2. [Configure Lakehouse Catalogs](/private-cloud/v2/configure-private-cloud/private-preview/ursa-lakehouse/configure-lakehouse-catalogs) -- Connect catalogs to the compaction service
3. [Enable Lakehouse Integration](/private-cloud/v2/configure-private-cloud/private-preview/ursa-lakehouse/enable-lakehouse-integration) -- Enable at cluster, namespace, or topic level


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.