Audit logs track and store authorization activities in Pulsar clusters, tenants, namespaces, and topics. After a Pulsar cluster is up and running within a large team, it’s critical to keep an eye on who is touching data and what they’re doing with it. Structured audit logs provide an easy way to track user/application access, so you can identify potential anomalies and bad actors.
Structured audit logs enable you to capture audit logs in a set of dedicated Pulsar topics, either on a local or a remote cluster, including:
Because the audit logs are stored in a Pulsar topic, you should configure the cleanup and backlog policy for the namespace to avoid having the Pulsar retention policies inadvertently clean up the audit logs. For more information, see Configure policies for a namespace.
If you did not enable audit log when you set up your cluster, or if you want to change the settings, follow these steps.
To verify that audit log is running, on the left navigation pane of StreamNative Cloud Console, in the Admin area, click Pulsar Clusters. You can verify that the audit log is running, as shown in the following figure.
Each audit log includes information about the event, event time, and permission status. The supported audit event types include:
Category | Type | Event type | Description | Default value |
---|---|---|---|---|
Management | Cluster | CreateCluster | Create a Pulsar cluster. | true |
Cluster | UpdateCluster | Update Pulsar cluster information. | true | |
Cluster | DeleteCluster | Delete Pulsar cluster information. | true | |
Tenant | CreateTenant | Create a Pulsar tenant. | true | |
Tenant | UpdateTenant | Update tenant information. | true | |
Tenant | DeleteTenant | Delete a Pulsar tenant. | true | |
Namespace | CreateNamespace | Create a namespace. | true | |
Namespace | DeleteNamespace | Delete a namespace. | true | |
Topic | CreatePartitionedTopic | Create a partitioned topic. | true | |
Topic | UpdatePartitions | Update partitions for a partitioned topic. | true | |
Topic | DeletePartitionedTopic | Delete a partitioned topic. | true | |
Subscription | CreateSubscription | Create a subscription. | true | |
Subscription | DeleteSubscription | Delete a subscription. | true | |
Describe | Cluster | ListClusters | List Pulsar clusters. | false |
Cluster | GetCluster | Get cluster information. | false | |
Tenant | ListTenants | List Pulsar tenants. | false | |
Tenant | GetTenant | Get tenant information. | false | |
Namespace | ListNamespaces | List Pulsar namespaces. | false | |
Namespace | GetNamespace | Get namespace information. | false | |
Topic | ListTopics | List Pulsar topics. | false | |
Topic | ListPartitionedTopics | List partitioned Pulsar topics. | false | |
Topic | GetPartitions | Get partitions of a partitioned topic. | false | |
Subscription | ListSubscriptions | List subscriptions of a topic. | false | |
Produce | Producer | NewProducer | Create a producer to produce messages to the topic. | false |
Producer | CloseProducer | Close a producer. | false | |
Consume | Consumer | NewConsumer | Create a consumer to subscribe to the topic. | false |
Consumer | CloseConsumer | Close a consumer. | false |
You can use Pulsar clients, Pulsar CLI, Rest API, and sink connectors to process and analyze the audit events stored in the Pulsar topics.
You can get data from the audit log topic with Pulsar clients. Refer to the Connect section to learn how to configure Pulsar clients for StreamNative Cloud cluster and consume data from the audit log topic.
The pulsar-client
is also a helpful tool to get data from the audit log topic.
StreamNative Cloud supports Rest API, which provides a RESTful interface to a Pulsar cluster. For more information, see Rest API to learn how to leverage the Rest API to consume data from the audit log topic.
If you want to integrate audit log data with your other data systems like Google BigQuery, AWS SQS, and AWS Kinesis, use a sink connector. For more information, see Deploy connectors to learn how to leverage connectors to integrate and synchronize audit log data.