# StreamNative Documentation: StreamNative Cloud Kafka ## Kafka ### Introduction - [StreamNative Kafka Service](https://docs.streamnative.io/kafka/overview.md): Native Apache Kafka service powered by the Ursa Engine. Leaderless, lakehouse-native, and up to 95% lower cost. - [Kafka Cluster vs. KSN on Pulsar Clusters](https://docs.streamnative.io/kafka/kafka-cluster-vs-ksn.md): Compare native Kafka Clusters with Kafka compatibility on Pulsar Clusters (KSN). Understand when to use each option. ### Get Started - [QuickStart - Kafka Client](https://docs.streamnative.io/cloud/get-started/quickstart-kafka.md) - [Get Started with Kafka Service](https://docs.streamnative.io/kafka/kafka-getting-started.md): Start producing and consuming messages with StreamNative Kafka Service in under 5 minutes. ### Kafka Clusters - [Manage Kafka Clusters](https://docs.streamnative.io/kafka/kafka-cluster-guide.md): Create, configure, and manage Kafka clusters on StreamNative Cloud with the Ursa Engine. ### Process - [Kafka Streams & KSQL](https://docs.streamnative.io/cloud/process/kafka-streams-and-ksql/kafka-streams-and-ksql.md) - [Connect to your cluster using the Kafka Stream](https://docs.streamnative.io/cloud/process/kafka-streams-and-ksql/cloud-connect-kafka-stream.md) - [Connect to your cluster using KSQL](https://docs.streamnative.io/cloud/process/kafka-streams-and-ksql/cloud-connect-ksql.md) ### Lakehouse - [Overview - Lakehouse Tables](https://docs.streamnative.io/cloud/lakehouse/lakehouse-tables-overview.md) - [Lakehouse Table Overview](https://docs.streamnative.io/cloud/lakehouse/lakehouse-table-overview.md) - [Enable Lakehouse Table](https://docs.streamnative.io/cloud/lakehouse/enable-lakehouse-integration.md) - [Lakehouse Observability](https://docs.streamnative.io/cloud/lakehouse/lakehouse-observability.md) #### Lakehouse Features - [Lakehouse Features Overview](https://docs.streamnative.io/cloud/lakehouse/lakehouse-features.md) - [Schema Evolution](https://docs.streamnative.io/cloud/lakehouse/features/schema-evolution.md) - [Variant Type](https://docs.streamnative.io/cloud/lakehouse/features/variant-type.md) - [Partition Key](https://docs.streamnative.io/cloud/lakehouse/features/iceberg-partition-key.md) - [Upsert](https://docs.streamnative.io/cloud/lakehouse/features/iceberg-upsert.md) - [Persist Key](https://docs.streamnative.io/cloud/lakehouse/features/persist-key.md) - [Persist Extra Metadata](https://docs.streamnative.io/cloud/lakehouse/features/persist-extra-metadata.md) #### Internal Tables - [Overview - Internal Tables](https://docs.streamnative.io/cloud/lakehouse/internal-tables/internal-tables-overview.md) #### External Tables - [Overview - External Tables](https://docs.streamnative.io/cloud/lakehouse/external-tables/external-tables-overview.md) #### Catalogs - [Overview of StreamNative Catalogs](https://docs.streamnative.io/cloud/lakehouse/catalogs/catalog-overview.md) - [Register Catalog](https://docs.streamnative.io/cloud/lakehouse/catalogs/register-catalog.md) - [Manage StreamNative Catalogs](https://docs.streamnative.io/cloud/lakehouse/catalogs/manage-catalogs.md) ##### Prepare Catalogs - [Prepare Lakehouse Catalogs](https://docs.streamnative.io/cloud/lakehouse/prepare-lakehouse-catalogs.md) - [BigLake for Iceberg](https://docs.streamnative.io/cloud/lakehouse/prepare-catalogs/biglake/iceberg.md) - [Horizon Catalog for Iceberg on AWS](https://docs.streamnative.io/cloud/lakehouse/prepare-catalogs/horizon-catalog/iceberg/aws.md) - [S3Table for Iceberg](https://docs.streamnative.io/cloud/lakehouse/prepare-catalogs/s3table/iceberg.md) ###### Open Catalog (Iceberg) - [Open Catalog for Iceberg on AWS](https://docs.streamnative.io/cloud/lakehouse/prepare-catalogs/open-catalog/iceberg/aws.md) - [Open Catalog for Iceberg on Azure](https://docs.streamnative.io/cloud/lakehouse/prepare-catalogs/open-catalog/iceberg/azure.md) - [Open Catalog for Iceberg on GCP](https://docs.streamnative.io/cloud/lakehouse/prepare-catalogs/open-catalog/iceberg/gcp.md) ###### Unity Catalog (Iceberg) - [Unity Catalog Managed Iceberg Table on AWS](https://docs.streamnative.io/cloud/lakehouse/prepare-catalogs/unity-catalog/iceberg/aws.md) - [Unity Catalog Managed Iceberg Table on Azure](https://docs.streamnative.io/cloud/lakehouse/prepare-catalogs/unity-catalog/iceberg/azure.md) - [Unity Catalog Managed Iceberg Table on GCP](https://docs.streamnative.io/cloud/lakehouse/prepare-catalogs/unity-catalog/iceberg/gcp.md) ###### Unity Catalog (Delta Lake) - [Unity Catalog for Delta Lake on AWS](https://docs.streamnative.io/cloud/lakehouse/prepare-catalogs/unity-catalog/delta-lake/aws.md) - [Unity Catalog for Delta Lake on Azure](https://docs.streamnative.io/cloud/lakehouse/prepare-catalogs/unity-catalog/delta-lake/azure.md) - [Unity Catalog for Delta Lake on GCP](https://docs.streamnative.io/cloud/lakehouse/prepare-catalogs/unity-catalog/delta-lake/gcp.md) ##### Private Networking - [Private Networking for Catalog Integration](https://docs.streamnative.io/cloud/lakehouse/catalogs/private-networking/overview.md) - [Private Networking for Snowflake Open Catalog](https://docs.streamnative.io/cloud/lakehouse/catalogs/private-networking/snowflake-open-catalog.md) - [Private Networking for Snowflake Horizon Catalog](https://docs.streamnative.io/cloud/lakehouse/catalogs/private-networking/snowflake-horizon-catalog.md) - [Private Networking for Databricks Unity Catalog (Delta Lake)](https://docs.streamnative.io/cloud/lakehouse/catalogs/private-networking/databricks-unity-catalog.md) - [Private Networking for Databricks Unity Catalog (Iceberg)](https://docs.streamnative.io/cloud/lakehouse/catalogs/private-networking/databricks-iceberg.md) - [Private Networking for Amazon S3 Tables](https://docs.streamnative.io/cloud/lakehouse/catalogs/private-networking/s3-tables.md) - [Private Networking for Google BigLake](https://docs.streamnative.io/cloud/lakehouse/catalogs/private-networking/google-biglake.md) ### Governance - [Kafka Schema Registry](https://docs.streamnative.io/kafka/governance/sr/overview.md) - [Connect to the Kafka Schema Registry](https://docs.streamnative.io/kafka/governance/sr/connect.md): Find your Schema Registry endpoint, authenticate, and configure Kafka clients to register and resolve schemas. - [Kafka Schema Registry Tutorial](https://docs.streamnative.io/kafka/governance/sr/tutorial.md): Register a schema, produce and consume typed records, then evolve the schema—watch an incompatible change fail and fix it. - [Kafka Schema Registry FAQ](https://docs.streamnative.io/kafka/governance/sr/faq.md): Common questions about the Kafka Schema Registry on StreamNative Cloud, including migration from Confluent. #### Fundamentals - [Kafka Schema Registry Key Concepts](https://docs.streamnative.io/kafka/governance/sr/fundamentals/key-concepts.md): Subjects, versions, schema IDs, and the record wire format—the vocabulary you need before working with the Kafka Schema Registry. - [Kafka Schema Evolution and Compatibility](https://docs.streamnative.io/kafka/governance/sr/fundamentals/schema-evolution.md): The compatibility modes the Kafka Schema Registry enforces, what each one permits, and the order in which to upgrade producers and consumers. - [Kafka Serializers and Deserializers](https://docs.streamnative.io/kafka/governance/sr/fundamentals/serdes.md): How Kafka serializers register and resolve schemas, how subject names are derived, and what auto-registration and normalization actually do. - [Use Kafka Schema References](https://docs.streamnative.io/kafka/governance/sr/fundamentals/schema-references.md): Compose schemas from other registered schemas so shared types are defined once, and understand how references affect deletion. - [Secure Kafka Schema Registry Access](https://docs.streamnative.io/kafka/governance/sr/fundamentals/security.md): Authentication mechanisms, per-subject authorization, and network exposure for the Kafka Schema Registry on StreamNative Cloud. ##### Schema Formats - [Use Avro Schemas](https://docs.streamnative.io/kafka/governance/sr/fundamentals/formats/avro.md): Configure the Avro serializer and deserializer, choose between SpecificRecord and GenericRecord, and understand Avro's compatibility rules. - [Use Protobuf Schemas](https://docs.streamnative.io/kafka/governance/sr/fundamentals/formats/protobuf.md): Configure the Protobuf serializer and deserializer, handle null values, use imports as schema references, and understand Protobuf's compatibility rules. - [Use JSON Schemas](https://docs.streamnative.io/kafka/governance/sr/fundamentals/formats/json-schema.md): Configure the JSON Schema serializer and deserializer, understand open and closed content models, and work through the compatibility rules per construct. #### Manage Schemas - [Register and Manage Kafka Schemas](https://docs.streamnative.io/kafka/governance/sr/manage/manage-schemas.md): Register schemas, inspect subjects and versions, set compatibility modes, and adopt a workflow that keeps schema changes deliberate. - [Delete Kafka Schemas](https://docs.streamnative.io/kafka/governance/sr/manage/delete-schemas.md): Soft delete, hard delete, what blocks a deletion, and what you can't get back. - [Validate Schema IDs on the Broker](https://docs.streamnative.io/kafka/governance/sr/manage/schema-id-validation.md): Configure the broker to reject records whose schema ID isn't registered for the topic's subject, so bad data never reaches the log. - [Set up active-standby replication for Kafka Schema Registry using Universal Linking](https://docs.streamnative.io/cookbook/kafka-schema-registry-geo-replication.md): Replicate schemas to one or more standby clusters with Universal Linking so consumers and producers stay online across availability zones. #### Reference - [Kafka Schema Registry REST API Examples](https://docs.streamnative.io/kafka/governance/sr/reference/rest-api.md): Worked curl examples for every supported Schema Registry endpoint, with request and response. - [Kafka Schema Registry Client Configuration](https://docs.streamnative.io/kafka/governance/sr/reference/client-configuration.md): The Schema Registry client properties StreamNative Cloud honors—connection, authentication, serializer behavior, and caching. - [Confluent Schema Registry API Compatibility](https://docs.streamnative.io/kafka/governance/sr/reference/confluent-compatibility.md): Which Confluent Schema Registry endpoints and features StreamNative Cloud implements, which it doesn't, and where behavior differs. - [Troubleshoot the Kafka Schema Registry](https://docs.streamnative.io/kafka/governance/sr/reference/troubleshooting.md): Error codes, what causes them, and how to fix them—plus the silent failures that produce no error at all. ### Migrate - [Migrate to StreamNative Kafka Service](https://docs.streamnative.io/kafka/kafka-migration-guide.md): Migrate from Amazon MSK, Confluent Cloud, or self-managed Apache Kafka to StreamNative Kafka Service with zero code changes. ### Kafka Clients - [Build Kafka Client Applications on StreamNative Cloud](https://docs.streamnative.io/cloud/build/kafka-clients/kafka-on-cloud.md) - [Kafka Rest API Quickstart](https://docs.streamnative.io/cloud/build/kafka-clients/kafka-rest-api.md) - [Migrating to StreamNative](https://docs.streamnative.io/cloud/build/kafka-clients/migrating-to-streamnative.md) - [Test Kafka Protocol as code](https://docs.streamnative.io/cloud/build/kafka-clients/test-kafka-protocol.md) #### Compatibility - [Kafka Compatibility Overview](https://docs.streamnative.io/cloud/build/kafka-clients/compatibility/kafka-compatibility.md) - [Kafka Protocol and Features](https://docs.streamnative.io/cloud/build/kafka-clients/compatibility/kafka-protocol-and-features.md) #### Quick Starts - [Connect to your cluster using the Kafka Java client](https://docs.streamnative.io/cloud/build/kafka-clients/quick-starts/cloud-connect-kafka-java.md) - [Connect to your cluster using the Kafka Python client](https://docs.streamnative.io/cloud/build/kafka-clients/quick-starts/cloud-connect-kafka-python.md) - [Connect to your cluster using the Kafka Go client](https://docs.streamnative.io/cloud/build/kafka-clients/quick-starts/cloud-connect-kafka-go.md) - [Connect to your cluster using the Kafka Node.js client](https://docs.streamnative.io/cloud/build/kafka-clients/quick-starts/cloud-connect-kafka-nodejs.md) - [Connect to your cluster using the Kafka C client (rdkafka)](https://docs.streamnative.io/cloud/build/kafka-clients/quick-starts/cloud-connect-kafka-c.md) - [Connect to your cluster using Kafka CLI](https://docs.streamnative.io/cloud/build/kafka-clients/quick-starts/cloud-connect-kafka-cli.md) #### Advanced Features - [Multi-Tenancy](https://docs.streamnative.io/cloud/build/kafka-clients/advanced-features/kafka-multi-tenancy.md) - [Kafka Compacted Topic](https://docs.streamnative.io/cloud/build/kafka-clients/advanced-features/kafka-compacted-topic.md) - [Kafka Transactions](https://docs.streamnative.io/cloud/build/kafka-clients/advanced-features/kafka-transaction.md) - [Interoperability between Apache Kafka and Apache Pulsar](https://docs.streamnative.io/cloud/build/kafka-clients/advanced-features/interoperability-between-kafka-and-pulsar.md) #### Configuration - [Configuring Kafka Clients](https://docs.streamnative.io/cloud/build/kafka-clients/config/config-kafka-client.md) - [Configuring Kafka Consumer](https://docs.streamnative.io/cloud/build/kafka-clients/config/config-kafka-consumer.md) - [Configuring Kafka Producer](https://docs.streamnative.io/cloud/build/kafka-clients/config/config-kafka-producer.md) #### Optimize And Tune - [Optimize and Tune Kafka Clients](https://docs.streamnative.io/cloud/build/kafka-clients/optimize-and-tune/optimize-kafka-clients.md) - [Optimize Kafka Client for Throughput](https://docs.streamnative.io/cloud/build/kafka-clients/optimize-and-tune/optimize-kafka-throughput.md) - [Optimize Kafka Client for Latency](https://docs.streamnative.io/cloud/build/kafka-clients/optimize-and-tune/optimize-kafka-latency.md) - [Optimize Kafka Client for Durability](https://docs.streamnative.io/cloud/build/kafka-clients/optimize-and-tune/optimize-kafka-durability.md) - [Optimize Kafka Client for Availability](https://docs.streamnative.io/cloud/build/kafka-clients/optimize-and-tune/optimize-kafka-availability.md) #### Integrations - [Connect to your cluster using Lenses](https://docs.streamnative.io/cloud/build/kafka-clients/integrations/cloud-connect-lenses.md) #### Guides - [Connect to StreamNative Cloud Using Kafka Clients](https://docs.streamnative.io/clients/kafka-clients/kafka-clients-overview.md) ##### Java - [Kafka Java Client Guide](https://docs.streamnative.io/clients/kafka-clients/java/kafka-java-client-guide.md) ###### Tutorial - [Introduction](https://docs.streamnative.io/clients/kafka-clients/java/tutorial/kafka-java-introduction.md) - [Prerequisites](https://docs.streamnative.io/clients/kafka-clients/java/tutorial/kafka-java-prerequisites.md) - [Create Project](https://docs.streamnative.io/clients/kafka-clients/java/tutorial/kafka-java-create-project.md) - [Cluster Setup](https://docs.streamnative.io/clients/kafka-clients/java/tutorial/kafka-java-cluster-setup.md) - [Create Topic](https://docs.streamnative.io/clients/kafka-clients/java/tutorial/kafka-java-create-topic.md) - [Build Producer](https://docs.streamnative.io/clients/kafka-clients/java/tutorial/kafka-java-build-producer.md) - [Build Consumer](https://docs.streamnative.io/clients/kafka-clients/java/tutorial/kafka-java-build-consumer.md) - [Produce Messages](https://docs.streamnative.io/clients/kafka-clients/java/tutorial/kafka-java-produce-messages.md) - [Consume Messages](https://docs.streamnative.io/clients/kafka-clients/java/tutorial/kafka-java-consume-messages.md) - [What's Next](https://docs.streamnative.io/clients/kafka-clients/java/tutorial/kafka-java-whats-next.md) ##### .net ###### Tutorial - [Introduction](https://docs.streamnative.io/clients/kafka-clients/.net/tutorial/kafka-dotnet-introduction.md) - [Prerequisites](https://docs.streamnative.io/clients/kafka-clients/.net/tutorial/kafka-dotnet-prerequisites.md) - [Create Project](https://docs.streamnative.io/clients/kafka-clients/.net/tutorial/kafka-dotnet-create-project.md) - [Cluster Setup](https://docs.streamnative.io/clients/kafka-clients/.net/tutorial/kafka-dotnet-cluster-setup.md) - [Create Topic](https://docs.streamnative.io/clients/kafka-clients/.net/tutorial/kafka-dotnet-create-topic.md) - [Build Producer](https://docs.streamnative.io/clients/kafka-clients/.net/tutorial/kafka-dotnet-build-producer.md) - [Build Consumer](https://docs.streamnative.io/clients/kafka-clients/.net/tutorial/kafka-dotnet-build-consumer.md) - [Produce Messages](https://docs.streamnative.io/clients/kafka-clients/.net/tutorial/kafka-dotnet-produce-messages.md) - [Consume Messages](https://docs.streamnative.io/clients/kafka-clients/.net/tutorial/kafka-dotnet-consume-messages.md) - [What's Next](https://docs.streamnative.io/clients/kafka-clients/.net/tutorial/kafka-dotnet-whats-next.md) ##### C And C++ ###### Tutorial - [Introduction](https://docs.streamnative.io/clients/kafka-clients/c-and-c++/tutorial/kafka-c-introduction.md) - [Prerequisites](https://docs.streamnative.io/clients/kafka-clients/c-and-c++/tutorial/kafka-c-prerequisites.md) - [Create Project](https://docs.streamnative.io/clients/kafka-clients/c-and-c++/tutorial/kafka-c-create-project.md) - [Cluster Setup](https://docs.streamnative.io/clients/kafka-clients/c-and-c++/tutorial/kafka-c-cluster-setup.md) - [Create Topic](https://docs.streamnative.io/clients/kafka-clients/c-and-c++/tutorial/kafka-c-create-topic.md) - [Build Producer](https://docs.streamnative.io/clients/kafka-clients/c-and-c++/tutorial/kafka-c-build-producer.md) - [Build Consumer](https://docs.streamnative.io/clients/kafka-clients/c-and-c++/tutorial/kafka-c-build-consumer.md) - [Produce Messages](https://docs.streamnative.io/clients/kafka-clients/c-and-c++/tutorial/kafka-c-produce-messages.md) - [Consume Messages](https://docs.streamnative.io/clients/kafka-clients/c-and-c++/tutorial/kafka-c-consume-messages.md) - [What's Next](https://docs.streamnative.io/clients/kafka-clients/c-and-c++/tutorial/kafka-c-whats-next.md) ##### Go ###### Tutorial - [Introduction](https://docs.streamnative.io/clients/kafka-clients/go/tutorial/kafka-go-introduction.md) - [Prerequisites](https://docs.streamnative.io/clients/kafka-clients/go/tutorial/kafka-go-prerequisites.md) - [Create Project](https://docs.streamnative.io/clients/kafka-clients/go/tutorial/kafka-go-create-project.md) - [Cluster Setup](https://docs.streamnative.io/clients/kafka-clients/go/tutorial/kafka-go-cluster-setup.md) - [Create Topic](https://docs.streamnative.io/clients/kafka-clients/go/tutorial/kafka-go-create-topic.md) - [Build Producer](https://docs.streamnative.io/clients/kafka-clients/go/tutorial/kafka-go-build-producer.md) - [Build Consumer](https://docs.streamnative.io/clients/kafka-clients/go/tutorial/kafka-go-build-consumer.md) - [Produce Messages](https://docs.streamnative.io/clients/kafka-clients/go/tutorial/kafka-go-produce-messages.md) - [Consume Messages](https://docs.streamnative.io/clients/kafka-clients/go/tutorial/kafka-go-consume-messages.md) - [What's Next](https://docs.streamnative.io/clients/kafka-clients/go/tutorial/kafka-go-whats-next.md) ##### Node.js ###### Tutorial - [Introduction](https://docs.streamnative.io/clients/kafka-clients/node.js/tutorial/kafka-js-introduction.md) - [Prerequisites](https://docs.streamnative.io/clients/kafka-clients/node.js/tutorial/kafka-js-prerequisites.md) - [Create Project](https://docs.streamnative.io/clients/kafka-clients/node.js/tutorial/kafka-js-create-project.md) - [Cluster Setup](https://docs.streamnative.io/clients/kafka-clients/node.js/tutorial/kafka-js-cluster-setup.md) - [Create Topic](https://docs.streamnative.io/clients/kafka-clients/node.js/tutorial/kafka-js-create-topic.md) - [Build Producer](https://docs.streamnative.io/clients/kafka-clients/node.js/tutorial/kafka-js-build-producer.md) - [Build Consumer](https://docs.streamnative.io/clients/kafka-clients/node.js/tutorial/kafka-js-build-consumer.md) - [Produce Messages](https://docs.streamnative.io/clients/kafka-clients/node.js/tutorial/kafka-js-produce-messages.md) - [Consume Messages](https://docs.streamnative.io/clients/kafka-clients/node.js/tutorial/kafka-js-consume-messages.md) - [What's Next](https://docs.streamnative.io/clients/kafka-clients/node.js/tutorial/kafka-js-whats-next.md) ##### Python ###### Tutorial - [Introduction](https://docs.streamnative.io/clients/kafka-clients/python/tutorial/kafka-python-introduction.md) - [Prerequisites](https://docs.streamnative.io/clients/kafka-clients/python/tutorial/kafka-python-prerequisites.md) - [Create Project](https://docs.streamnative.io/clients/kafka-clients/python/tutorial/kafka-python-create-project.md) - [Cluster Setup](https://docs.streamnative.io/clients/kafka-clients/python/tutorial/kafka-python-cluster-setup.md) - [Create Topic](https://docs.streamnative.io/clients/kafka-clients/python/tutorial/kafka-python-create-topic.md) - [Build Producer](https://docs.streamnative.io/clients/kafka-clients/python/tutorial/kafka-python-build-producer.md) - [Build Consumer](https://docs.streamnative.io/clients/kafka-clients/python/tutorial/kafka-python-build-consumer.md) - [Produce Messages](https://docs.streamnative.io/clients/kafka-clients/python/tutorial/kafka-python-produce-messages.md) - [Consume Messages](https://docs.streamnative.io/clients/kafka-clients/python/tutorial/kafka-python-consume-messages.md) - [What's Next](https://docs.streamnative.io/clients/kafka-clients/python/tutorial/kafka-python-whats-next.md) ##### Spring Boot ###### Tutorial - [Introduction](https://docs.streamnative.io/clients/kafka-clients/spring-boot/tutorial/kafka-spring-boot-introduction.md) - [Prerequisites](https://docs.streamnative.io/clients/kafka-clients/spring-boot/tutorial/kafka-spring-boot-prerequisites.md) - [Create Project](https://docs.streamnative.io/clients/kafka-clients/spring-boot/tutorial/kafka-spring-boot-create-project.md) - [Cluster Setup](https://docs.streamnative.io/clients/kafka-clients/spring-boot/tutorial/kafka-spring-boot-cluster-setup.md) - [Create Topic](https://docs.streamnative.io/clients/kafka-clients/spring-boot/tutorial/kafka-spring-boot-create-topic.md) - [Build Producer](https://docs.streamnative.io/clients/kafka-clients/spring-boot/tutorial/kafka-spring-boot-build-producer.md) - [Build Consumer](https://docs.streamnative.io/clients/kafka-clients/spring-boot/tutorial/kafka-spring-boot-build-consumer.md) - [Produce Messages](https://docs.streamnative.io/clients/kafka-clients/spring-boot/tutorial/kafka-spring-boot-produce-messages.md) - [Consume Messages](https://docs.streamnative.io/clients/kafka-clients/spring-boot/tutorial/kafka-spring-boot-consume-messages.md) - [What's Next](https://docs.streamnative.io/clients/kafka-clients/spring-boot/tutorial/kafka-spring-boot-whats-next.md)