Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.streamnative.io/llms.txt

Use this file to discover all available pages before exploring further.

Use Universal Linking to replicate schemas from an active Kafka Schema Registry to one or more standby clusters asynchronously. This lets you maintain schema availability across multiple availability zones so that consumers and producers can still read schemas when the active cluster is temporarily unavailable.

​
How it works

UniLinkSchema is one of the replication jobs offered by Universal Linking to replicate schemas between clusters. For each standby cluster, a UniLinkSchema resource runs a replication job that pulls schema changes from the active cluster’s Schema Registry every minute and applies them to the standby cluster. The standby cluster’s Schema Registry operates in import mode, which means it accepts replicated schemas but doesn’t allow new schema registrations from producers. To give users a single stable Schema Registry endpoint, you must configure your own Schema Registry proxy — such as Envoy, HAProxy, NGINX, or another HTTP traffic server — to route traffic to whichever cluster is currently active. StreamNative Cloud doesn’t create or manage this proxy for you. Key constraints:
  • Only one cluster can be the active Schema Registry at a time.
  • Standby clusters provide read-only access to schemas.
  • You can extend the setup to multiple standby clusters by creating one UniLinkSchema resource per standby.

​
Limitations

Active-active replication is not supported. UniLinkSchema only supports one-directional (active → standby) replication.Deletions are not replicated. If a subject or schema version is deleted (soft delete or hard delete) on the active cluster, the deletion is not synced to standby clusters.

​
Prerequisites

  • One active cluster as the source
  • One or more standby clusters as destinations
  • Permission to create UniLinkSchema and Secret resources in your StreamNative Cloud organization

​
Part 1: Set up active-standby replication

​
Step 1: Get the Schema Registry URL of the active cluster

The Schema Registry URL depends on the cluster type:
  • Pulsar clusters with Kafka protocol enabled (KSN): The Schema Registry URL appends /kafka to the cluster’s HTTP service URL. The /kafka sub-path is specific to StreamNative’s Kafka Schema Registry running on Pulsar clusters.
  • Kafka clusters (UFK): The Schema Registry URL uses the cluster’s HTTP service URL directly without a sub-path.
For Pulsar clusters with Kafka protocol enabled, get the Schema Registry URL:
snctl get pulsarcluster <cluster-name> \
  -o jsonpath='https://{.spec.serviceEndpoints[?(@.type=="service")].dnsName}/kafka{"\n"}'
The resulting URL follows this format:
https://<pulsar-cluster-dns-name>/kafka

​
Step 2: Get an administrator API key for the active cluster

You need an API key with administrator permissions on the active cluster’s Schema Registry. Follow Create an API key to create an API key, and save the generated API key token for the next step.

​
Step 3: Create a secret for UniLinkSchema credentials

Create a Secret resource that UniLinkSchema uses to authenticate against the active cluster’s Schema Registry.
Example YAML:
apiVersion: cloud.streamnative.io/v1alpha1
kind: Secret
metadata:
  name: schema-sync
  namespace: <org>
data:
  apiKey: public
  apiSecret: <admin-api-key-token-from-step-2>
Fields:
FieldDescription
data.apiKeyAny non-empty string (for example, public)
data.apiSecretThe administrator API key token from Step 2
Save the YAML as schema-sync-secret.yaml, then apply it:
snctl apply -f schema-sync-secret.yaml

​
Step 4: Create the UniLinkSchema resource

Create a UniLinkSchema resource. This resource configures the active cluster as the source and the standby cluster as the destination.
UniLinkSchema creation isn’t yet available in StreamNative Console for schema geo replication. Use snctl to create this resource.
apiVersion: cloud.streamnative.io/v1alpha1
kind: UniLinkSchema
metadata:
  name: schema-replication-1
  namespace: <org>
spec:
  destClusterRef: <standby-cluster-name>
  sourceSchemaRegistry:
    url: <active-cluster-schema-registry-url>
    secretRef: <secret-name-from-step-3>
  subjectMappings:
    - subjects: ".*"
      destinationPrefix: ""
Save the YAML as schema-replication-1.yaml, then apply it:
snctl apply -f schema-replication-1.yaml
After you apply this resource:
  • The standby cluster’s Schema Registry switches to import mode: producers cannot auto-register new schemas, but existing schemas remain readable.
  • Schemas sync incrementally approximately every minute.
Verify the setup:Check the UniLinkSchema resource status:
snctl get unilinkschema schema-replication-1

​
Step 5: Add more standby clusters (optional)

To replicate to additional standby clusters, repeat Steps 1–4 for each standby. Create a separate Secret and UniLinkSchema resource for each destination:
  • active → standby-1 (schema-replication-1)
  • active → standby-2 (schema-replication-2)

​
Part 2: Perform a failover

When the active cluster is unavailable, promote a standby cluster to be the new active and rebuild replication from that cluster.

​
Step 1: Delete the existing UniLinkSchema resources

Delete the UniLinkSchema resources that point to the failed active cluster. This stops schema replication from that cluster and takes the standby clusters out of import mode. 
snctl delete unilinkschema schema-replication-1

​
Step 2: Create new UniLinkSchema resources from the new active cluster

Repeat the setup steps using the promoted standby cluster as the new source: 
apiVersion: cloud.streamnative.io/v1alpha1
kind: UniLinkSchema
metadata:
  name: schema-replication-2
  namespace: <org>
spec:
  destClusterRef: <new-standby-cluster>
  sourceSchemaRegistry:
    url: <new-active-cluster-schema-registry-url>
    secretRef: <new-active-cluster-secret>
  subjectMappings:
    - subjects: ".*"
      destinationPrefix: ""
Update your Schema Registry proxy to route traffic to the new active cluster’s Schema Registry URL.

​
What’s next