If you are using a Ursa-Engine powered cluster, please note that transactions and topic compaction are not supported in Ursa Engine.
StreamNative supports Apache Kafka® compatible transaction semantics and APIs. For example, you can fetch messages starting from the last consumed offset and process them transactionally one by one, updating the last consumed offset and generating events as you go.
If a producer sends multiple messages to the same or different partitions and a network connection or broker failure occurs, it can be guaranteed that all messages are either completely written to the partition or not at all. This is crucial for applications that require strict guarantees, such as financial services transactions.
Transactions ensure exactly-once semantics (EOS) and atomicity.
EOS helps developers avoid the anomalies of at-most-once processing (possible event loss) and at-least-once processing (possible event duplication). Combined with idempotent producers, StreamNative supports EOS, ensuring that events are neither lost nor duplicated. Atomicity commits a set of messages across partitions as a unit: either all messages are committed, or none are. Data encapsulated and transmitted in a single operation can only succeed or fail globally, ensuring consistent transaction outcomes.
This section describes how to connect to your cluster and send transaction messages.
kcat
will be equal to token:<API KEY>
.You can follow the instructions to create an API key for the service account you choose to use.
Add Maven dependencies.
Open a terminal and run a Kafka consumer to receive a message from the test-transaction-topic
topic. In this case, the isolation level to read_committed
, which means only 5 committed messages will be consumed, if you want to consume both committed and uncommitted messages, set it to read_uncommitted
.
serverUrl
: the Kafka service URL of your StreamNative cluster.jwtToken
: an API key of your service account.Open another terminal and run a Kafka producer to send 5 messages to the test-transaction-topic
topic and commit it, then send 5 messages and abort it.
serverUrl
: the Kafka service URL of your StreamNative cluster.jwtToken
: an API key of your service account.If you are using a Ursa-Engine powered cluster, please note that transactions and topic compaction are not supported in Ursa Engine.
StreamNative supports Apache Kafka® compatible transaction semantics and APIs. For example, you can fetch messages starting from the last consumed offset and process them transactionally one by one, updating the last consumed offset and generating events as you go.
If a producer sends multiple messages to the same or different partitions and a network connection or broker failure occurs, it can be guaranteed that all messages are either completely written to the partition or not at all. This is crucial for applications that require strict guarantees, such as financial services transactions.
Transactions ensure exactly-once semantics (EOS) and atomicity.
EOS helps developers avoid the anomalies of at-most-once processing (possible event loss) and at-least-once processing (possible event duplication). Combined with idempotent producers, StreamNative supports EOS, ensuring that events are neither lost nor duplicated. Atomicity commits a set of messages across partitions as a unit: either all messages are committed, or none are. Data encapsulated and transmitted in a single operation can only succeed or fail globally, ensuring consistent transaction outcomes.
This section describes how to connect to your cluster and send transaction messages.
kcat
will be equal to token:<API KEY>
.You can follow the instructions to create an API key for the service account you choose to use.
Add Maven dependencies.
Open a terminal and run a Kafka consumer to receive a message from the test-transaction-topic
topic. In this case, the isolation level to read_committed
, which means only 5 committed messages will be consumed, if you want to consume both committed and uncommitted messages, set it to read_uncommitted
.
serverUrl
: the Kafka service URL of your StreamNative cluster.jwtToken
: an API key of your service account.Open another terminal and run a Kafka producer to send 5 messages to the test-transaction-topic
topic and commit it, then send 5 messages and abort it.
serverUrl
: the Kafka service URL of your StreamNative cluster.jwtToken
: an API key of your service account.