Data Retention
Pulsar has a different data retention policy by default. In Pulsar, consumed and acknowledged data from all subscriptions, or data from topics with no subscriptions, is systematically removed from the topic segment by segment. In contrast, Kafka retains data within the topic for a fixed duration of 7 days, regardless of whether it has been consumed or not. However, StreamNative adopts Pulsar’s approach, offering users a data retention policy that can discern data consumption patterns. Therefore, when migrating to StreamNative, it is necessary to proactively adjust the data retention policy to prevent the deletion of data that has been written, after consumption, or in the absence of subscriptions. Certainly, if Pulsar’s behavior aligns with your expectations, there would be no need to modify the policy. You can follow the Pulsar Admin CLI, Pulsar Admin API or Pulsar Admin REST API to set the data retention policy for the namespace or topic. In practice, you can see that the retention policy is not configured in thepublic/default
namespace via the Pulsar admin CLI:
null
means the retention policy is not set. The default behavior is that all messages could be deleted after the retention period. You can set the retention policy for the namespace with the following command: