Migrating to KSN
KSN adds the native Kafka protocol to Pulsar brokers, but it does not mean KSN is an exact copy of Apache Kafka. As KSN not only maintains compatibility with the Kafka protocol but also incorporates many of the excellent features of Pulsar, it is crucial for Kafka users to understand these differences before migrating to KSN. By doing so, they can avoid any potential losses resulting from the disparities between the two systems.
Data Retention
Pulsar has a different data retention policy by default. In Pulsar, consumed and acknowledged data from all subscriptions or the topic has no subscriptions is systematically removed from the topic segment by segment. In Kafka, data within the topic is retained for a fixed duration of 7 days, regardless of whether it has been consumed or not. However, KSN employs the approach of Pulsar to offer users a data retention policy that is capable of discerning data consumption patterns. Therefore, when migrating to KSN, it is necessary to proactively adjust the data retention policy to prevent data that has been written from being deleted after consumption or in the absence of subscriptions. Certainly, if the behavior of Pulsar aligns with your expectations, there would be no need to modify the policy.
You can follow the Pulsar Admin CLI, Pulsar Admin API or Pulsar Admin REST API to set the data retention policy for the namespace or topic.