Use Protobuf Schemas

Protobuf brings its own compatibility model to the Schema Registry. Most of what makes a Protobuf change safe or unsafe comes from the field-numbering rules in the Protobuf specification rather than from the registry, so a schema that evolves correctly by Protobuf’s rules generally passes the registry’s check too.

Configure the serializer

props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, KafkaProtobufSerializer.class);
props.put(KafkaProtobufSerializerConfig.SCHEMA_REGISTRY_URL_CONFIG, schemaRegistryUrl);
props.put(KafkaProtobufSerializerConfig.BASIC_AUTH_CREDENTIALS_SOURCE, "USER_INFO");
props.put(KafkaProtobufSerializerConfig.USER_INFO_CONFIG, String.format("%s:%s", "any-user", apiKey));

And the deserializer, which needs the generated class:

props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, KafkaProtobufDeserializer.class);
props.put(KafkaProtobufDeserializerConfig.SCHEMA_REGISTRY_URL_CONFIG, schemaRegistryUrl);
props.put(KafkaProtobufDeserializerConfig.SPECIFIC_PROTOBUF_VALUE_TYPE, Order.class);

Add the dependency:

<dependency>
  <groupId>io.confluent</groupId>
  <artifactId>kafka-protobuf-serializer</artifactId>
  <version>7.5.0</version>
</dependency>

Define a message

syntax = "proto3";
package com.acme;

message Order {
  string id = 1;
  double total = 2;
  optional string coupon_code = 3;
}

Order order = Order.newBuilder()
    .setId("order-1")
    .setTotal(42.0)
    .build();
producer.send(new ProducerRecord<>("orders", order.getId(), order));

Handling null

proto3 has no null. A field that isn’t set reads back as its zero value—0, "", false—which means you can’t distinguish “not provided” from “explicitly zero” unless you ask for it. Use optional (the recommended approach, available in protoc 3.15 and later):

optional string coupon_code = 3;

if (order.hasCouponCode()) {
    // The field was explicitly set
}

Use a wrapper type if you’re on an older protoc:

import "google/protobuf/wrappers.proto";

google.protobuf.StringValue coupon_code = 3;

Wrappers work but add a level of nesting to every access. Prefer optional on any new schema.

Compatibility rules

Field numbers are identity in Protobuf. Names are labels; numbers are what’s on the wire.

Change	Safe
Add a field with a new number	Yes
Rename a field, keeping its number	Yes—the number is what matters
Delete a field and `reserved` its number	Yes
Delete a field and reuse its number later	No—this corrupts old data
Change a field’s type	Only within Protobuf’s compatible-type rules
Change a field number	No
Move a field into or out of a `oneof`	No

Always reserve a number you retire:

message Order {
  reserved 4;
  reserved "legacy_discount";

  string id = 1;
  double total = 2;
}

The compatibility matrix on Overview lists FORWARD, FORWARD_TRANSITIVE, FULL, and FULL_TRANSITIVE as unsupported for Protobuf. The API accepts those values on a Protobuf subject rather than rejecting them, but the combination isn’t validated or supported. Use BACKWARD or BACKWARD_TRANSITIVE for Protobuf subjects.

Schema references through imports

Protobuf’s import statement maps naturally onto schema references. The reference name is the import path:

syntax = "proto3";
package com.acme;

import "com/acme/address.proto";

message Customer {
  string name = 1;
  Address address = 2;
}

"references": [
  {"name": "com/acme/address.proto", "subject": "com.acme.Address", "version": 1}
]

Imports are registered recursively when auto-registration is on, so registering Customer also registers everything it imports. That’s convenient in development and worth turning off in production, where you want registration to be deliberate. See Auto-registration.

Multiple event types on one topic

Protobuf handles several message types on one topic more naturally than the other formats, through a wrapper with a oneof:

message OrderEvent {
  oneof event {
    OrderPlaced placed = 1;
    OrderShipped shipped = 2;
    OrderCancelled cancelled = 3;
  }
}

Alternatively, use RecordNameStrategy so each message type gets its own subject and evolves independently. See Multiple event types on one topic.

Normalization

?normalize=true performs real normalization for Protobuf.

Next steps

Avro

Defaults, aliases, and schema resolution.

JSON Schema

Open content models and per-construct compatibility.

Introduction

Get Started

Kafka Clusters

Process

Lakehouse

Governance

Migrate

Kafka Clients

Configure the serializer

Define a message

Handling null

Compatibility rules

Schema references through imports

Multiple event types on one topic

Normalization

Next steps

Avro

JSON Schema

​Configure the serializer

​Define a message

​Handling null

​Compatibility rules

​Schema references through imports

​Multiple event types on one topic

​Normalization

​Next steps

Avro

JSON Schema

Configure the serializer

Define a message

Handling null

Compatibility rules

Schema references through imports

Multiple event types on one topic

Normalization

Next steps