Skip to main content
Lakehouse integration can be enabled or disabled at the cluster, namespace, and topic levels using dynamic configuration. This allows fine-grained control over which topics produce lakehouse tables.
For the full set of dynamic configuration keys (including topic-level feature settings such as partitionKey and upsertModeEnabled), see the Dynamic Configuration Guide.

​
Table Modes

ModePropertyDescription
External Table (SDT)sdt.enabledDelivers data to external lakehouse catalogs. Supports upsert, partition key, and schema evolution.
Internal Table (SBT)sbt.enabledComing Soon. Managed tables with streaming reads + analytics on the same data.
Compaction task publishing is enabled when either sdt.enabled or sbt.enabled is true.

​
Configuration Key Format

All dynamic configuration keys must be prefixed with the cluster name. The cluster name is the value of clusterName in the Pulsar broker’s conf/broker.conf. See Finding the Cluster Name for how to retrieve it. For example, if the cluster name is private-cloud:
ScopeKey formatExample
Cluster<cluster>.cluster.<key>private-cloud.cluster.sdt.enabled
Namespace/Topic<cluster>.<key>private-cloud.sdt.enabled

​
Supported Dynamic Configuration Keys

​
Cluster Level

Applied via the sn/system namespace:
KeyDescription
<cluster>.cluster.sdt.enabledEnable/disable External Table (SDT) for the cluster
<cluster>.cluster.sbt.enabledEnable/disable Internal Table (SBT) — Coming Soon
<cluster>.cluster.sdt.catalog.nameDefault catalog name for SDT
<cluster>.cluster.tail.compact.data.visibility.interval.in.secondsData visibility delay override

​
Namespace and Topic Level

KeyDescription
<cluster>.sdt.enabledEnable/disable SDT for a namespace or topic
<cluster>.sbt.enabledEnable/disable SBT — Coming Soon
<cluster>.sdt.catalog.nameCatalog name for a namespace or topic
<cluster>.tail.compact.data.visibility.interval.in.secondsData visibility delay override

​
Cluster-Level Configuration

Cluster-level properties are stored in the sn/system namespace.

​
Enable SDT for the cluster

bin/pulsar-admin namespaces set-properties \
  -p private-cloud.cluster.sdt.enabled=true \
  sn/system

​
Set the default catalog

bin/pulsar-admin namespaces set-properties \
  -p private-cloud.cluster.sdt.catalog.name=my-catalog \
  sn/system

​
Namespace-Level Configuration

​
Enable SDT for a namespace

bin/pulsar-admin namespaces set-properties \
  -p private-cloud.sdt.enabled=true \
  <tenant>/<namespace>

​
Assign a catalog to a namespace

bin/pulsar-admin namespaces set-property \
  -k catalog.name -v <catalog-name> \
  <tenant>/<namespace>
Example: 
bin/pulsar-admin namespaces set-property \
  -k catalog.name -v polaris-prod \
  public/iceberg

​
Topic-Level Configuration

Topic-level properties override namespace-level settings.

​
Enable SDT for a topic

bin/pulsar-admin topics update-properties \
  -p private-cloud.sdt.enabled=true \
  persistent://<tenant>/<namespace>/<topic>

​
Assign a catalog to a topic

bin/pulsar-admin topics update-properties \
  -p catalog.name=<catalog-name> \
  persistent://<tenant>/<namespace>/<topic>
Example: 
bin/pulsar-admin topics update-properties \
  -p catalog.name=s3table-analytics \
  persistent://public/iceberg/events-topic

​
Configuration Override Priority

Settings at a more specific level override broader settings: 
Topic properties
    ↓ (overrides)
Namespace properties
    ↓ (overrides)
Cluster properties (sn/system)

​
Extending Valid Configuration Keys

To introduce additional dynamic configuration keys, set the following environment variable when starting the Pulsar broker: 
LAKEHOUSE_DYNAMIC_EXTRA_VALID_KEYS_IN_CONF_FILE=key1,key2,key3
Example: 
LAKEHOUSE_DYNAMIC_EXTRA_VALID_KEYS_IN_CONF_FILE=clusterSdtEnabled,clusterSbtEnabled,clusterSdtCatalogName

​
Next Steps