If you are using a Ursa-Engine powered cluster, please note that transactions and topic compaction are not supported in Ursa Engine.
- Apache Kafka supports a
delete+compact
retention policy, which can remove the record of an old key even if it is the latest value for that key and applies the segment-based data cleanup policy (1 day by default). However, StreamNative cannot remove compacted keys by retention time or size unless a tombstone (null value) message is written into the topic. - The topic config options
max.compaction.lag.ms
andmin.compaction.lag.ms
are not supported. - Topic compaction for Kafka currently cannot work with transactions, but support for this feature is planned for the future.
- In Kafka, a tombstone (a key with a null value) is retained for a period set by
delete.retention.ms
. In contrast, StreamNative/Pulsar removes the tombstone immediately. - StreamNative supports manually triggering compaction, whereas Kafka does not.
Use Compacted Topic
Create Compacted Topic
To create a compact topic, you can follow the CLI Tools tutorial and use the following command line to create the compact topic:Configure Compaction Policy
You can change thecompaction-threshold
policy to control how often compression is triggered (default 100MB) it specifies how large the topic backlog can grow before compaction is triggered, or you can manually trigger compaction using the Pulsar administrative API. For more information, see Topic Compaction Cookbook.