Configure API key authentication

API key authentication lets you issue and revoke credentials without distributing a token-signing key to applications. The API key service issues each credential as a signed JSON Web Token (JWT), and Pulsar validates the token and its revocation status when a client connects. For broker and proxy authentication, this procedure uses the token jti claim as the Pulsar role. The API key service returns the same value as token_id. Grant namespace permissions to this value, not to the token name or its sub claim. Console uses sub separately to identify its administrator, as described in Configure Console for API key management.

Before you begin

Install StreamNative Operator v0.20.0 or later. If your cluster uses an earlier Operator version, upgrade it before enabling API key authentication. Version v0.20.0 introduces the ApiKeys custom resource and the PulsarCoordinator.spec.authentication.apiKey field required by this procedure.
Deploy a Pulsar cluster with PulsarCoordinator, PulsarBroker, and PulsarProxy resources in the same namespace.
Use StreamNative Console v2.3.32 or later for this procedure. Upgrade an earlier Console version before configuring API key management.
Install kubectl, jq, and pulsarctl.
Make sure that you can update the cluster custom resources and read Secrets in the cluster namespace.

The examples use the following values:

Resource	Example value
Kubernetes namespace	`pulsar`
Pulsar cluster and coordinator name	`private-cloud`

Replace these values to match your environment. The ApiKeys resource and PulsarCoordinator resource must have the same name and namespace.

Deploy the API key service

Create an ApiKeys resource:

apiVersion: k8s.streamnative.io/v1alpha1
kind: ApiKeys
metadata:
  name: private-cloud
  namespace: pulsar
  labels:
    k8s.streamnative.io/coordinator-name: private-cloud
spec:
  replicas: 1
  image: streamnative/sn-api-keys-svc:v0.11.4
  hostname: http://private-cloud-apikeys.pulsar.svc.cluster.local:8081
  issuerPathPrefix: /
  brokerServiceUrl: pulsar://private-cloud-broker.pulsar.svc.cluster.local:6650
  config:
    server:
      audience: urn:sn:pulsar:pulsar:private-cloud
      currentKid: private-cloud
      allowCustomClaims: true
    backend:
      pulsarTopic: api_keys_api.example
    authenticator:
      enabled: true
      issuers:
        - http://private-cloud-apikeys.pulsar.svc.cluster.local:8081/
      acceptedAudience: urn:sn:pulsar:pulsar:private-cloud

Use the same value for spec.config.server.audience and spec.config.authenticator.acceptedAudience. The audience format used by the coordinator is urn:sn:pulsar:<namespace>:<coordinator-name>. allowCustomClaims lets an authenticated administrator set claims such as scope and sub when creating an API key. You can enable this option only when spec.config.server.multiTenant.enable is false or omitted.

Prepare the API key backend topic

The API key service stores its state in the topic configured by spec.config.backend.pulsarTopic. The short topic name in this example resolves to:

persistent://public/default/api_keys_api.example

Before you start the API key service, use one of the following options to make this topic available.

Enable automatic topic creation

Add the following settings to the PulsarBroker resource:

spec:
  config:
    custom:
      PULSAR_PREFIX_allowAutoTopicCreation: "true"
      PULSAR_PREFIX_allowAutoTopicCreationType: "non-partitioned"
      PULSAR_PREFIX_defaultNumPartitions: "1"

With this configuration, the API key service creates api_keys_api.example as a non-partitioned topic when it first accesses the topic.

Create the topic manually

If automatic topic creation is off for your cluster, create the non-partitioned topic before you start the API key service or enable API key authentication. Use an existing administrator credential:

kubectl exec -n pulsar private-cloud-toolset-0 -- /pulsar/bin/pulsarctl \
  --admin-service-url http://private-cloud-proxy:8080 \
  --token-file "" \
  --token "<existing-admin-token>" \
  topics create \
  persistent://public/default/api_keys_api.example \
  0

The final 0 creates a non-partitioned topic. The administrator credential must have permission to manage the public/default namespace. If the API key service credential isn’t an administrator, grant it produce and consume permissions on this topic.

For a first-time deployment when automatic topic creation is off, create the backend topic before switching all brokers to API-key-dependent authentication. Otherwise, the brokers can wait for authentication metadata from the API key service while the API key service waits for its Pulsar backend topic.

Don’t expose the API key service directly to public networks. In this procedure, administrators manage API keys through Console.

Save the resource as apikeys.yaml. Apply it together with the cluster configuration in the next section so that the API key service and Pulsar authentication can become ready together.

Enable API key authentication on the cluster

Add the following configuration to the PulsarCoordinator resource:

spec:
  authentication:
    apiKey:
      enabled: true
      subjectClaim: jti

The operator configures the broker and proxy to discover the API key service signing keys and revocation list. It also creates internal API keys for cluster components. For broker and proxy authentication, subjectClaim: jti makes each API key’s unique token ID its Pulsar role. Use this role when you grant permissions to a non-administrator API key. Configure authorization on the PulsarBroker resource:

spec:
  config:
    authorization:
      superUserRoles: "broker-admin,proxy-admin,admin"

With API key authentication enabled on the coordinator, the operator enables broker authorization and selects the API key authorization provider. Use spec.config.authorization for broker roles and preserve any existing superuser roles when updating superUserRoles. Configure the PulsarProxy resource to forward the original client credentials to the broker:

spec:
  config:
    custom:
      PULSAR_PREFIX_forwardAuthorizationCredentials: "true"

The operator enables authenticateOriginalAuthData when you configure proxy authentication. Merge these fields with the existing broker and proxy configuration. Preserve unrelated settings. Remove any spec.config.custom overrides for authorizationEnabled, authorizationProvider, superUserRoles, or proxyRoles, including their PULSAR_PREFIX_ forms, so they don’t override the typed authorization configuration. Apply apikeys.yaml and the updated coordinator, broker, and proxy resources. Wait for the API key service to become ready:

kubectl wait -n pulsar \
  --for=condition=Ready \
  pod -l cloud.streamnative.io/component=apikeys \
  --timeout=180s

If your deployment uses different labels, run kubectl get pods -n pulsar to identify the API key service Pod.

Apply the proxy role workaround

StreamNative Operator v0.20.16 has a known issue that prevents it from automatically adding the proxy API key ID to the broker’s superuser and proxy roles. Use the following temporary workaround on affected versions. Once you upgrade to a release that fixes automatic proxy role synchronization, the manual patch is no longer required.

The operator creates an internal proxy credential in the <coordinator-name>-apikeys-proxy-key Secret. Read only its jti value:

PROXY_JTI=$(kubectl get secret private-cloud-apikeys-proxy-key -n pulsar \
  -o json | jq -r '.data.jti | @base64d')

Add that value to spec.config.authorization.superUserRoles and spec.config.authorization.proxyRoles on the broker. The following command preserves existing roles and avoids duplicate entries:

kubectl patch pulsarbroker private-cloud -n pulsar --type=merge \
  -p "$(kubectl get pulsarbroker private-cloud -n pulsar -o json | \
    jq -c --arg role "$PROXY_JTI" '
      def add_role: split(",") + [$role] | map(select(length > 0)) | unique | join(",");
      {spec:{config:{authorization:{
        superUserRoles: ((.spec.config.authorization.superUserRoles // "") | add_role),
        proxyRoles: ((.spec.config.authorization.proxyRoles // "") | add_role)
      }}}}')"

Keep these role settings in your deployment manifests so that subsequent deployments preserve the workaround.

On affected Operator versions, repeat this workaround if the operator regenerates the proxy API key, and remove the previous jti from both role settings. Otherwise, non-administrator clients fail with an error that indicates the proxy doesn’t have a valid proxy role.

Wait for the broker and proxy Pods to become ready after the configuration rollout:

kubectl wait -n pulsar \
  --for=condition=Ready \
  pod -l cloud.streamnative.io/component=broker \
  --timeout=300s

kubectl wait -n pulsar \
  --for=condition=Ready \
  pod -l cloud.streamnative.io/component=proxy \
  --timeout=300s

Configure Console for API key management

Add the following configuration to the Console resource. Keep your existing spec.image value if it meets the Console version prerequisite:

spec:
  config:
    authentication:
      apiKey:
        serviceUrl: http://private-cloud-apikeys.pulsar.svc.cluster.local:8081/
        secretName: private-cloud-apikeys-console-key
        superRole: admin
        subjectClaim: sub

The subjectClaim values in the coordinator and Console configurations intentionally differ:

Setting	Applies to	Purpose
`PulsarCoordinator.spec.authentication.apiKey.subjectClaim: jti`	Broker and proxy authentication	Uses the unique API key ID as the Pulsar authorization role.
`Console.spec.config.authentication.apiKey.subjectClaim: sub`	Console login and authorization	Uses the token subject as the Console user identity and compares it with `superRole`.

Keep subjectClaim: sub in the Console configuration. The generated Console administrator token has sub: admin, which matches superRole: admin. Its jti is a unique API key ID and doesn’t match the configured superuser role. If you set the Console subjectClaim to jti, token validation can succeed during sign-in, but Console rejects subsequent operations because the jti doesn’t match superRole. The Console setting doesn’t change how brokers and proxies derive the Pulsar role from jti. The operator stores the Console administrator credential in the configured Secret. Treat this Secret as a superuser credential. Apply the updated Console resource and wait for its Pod to become ready.

Create API keys in Console

Retrieve the Console administrator token from the configured Secret:
kubectl get secret -n pulsar private-cloud-apikeys-console-key \ -o json | jq -r '.data.token | @base64d'
Copy the token from the command output. Don’t save it in a file or shell history.
Open Console, paste the administrator token into the JWT Token field, and then click Sign in.
Sign in to Console with the administrator token.
In the left navigation menu, select API Keys.

Choose an expiration time

When you create an API key, select one of the following values from Expiration date:

30 days
90 days
1 year
Custom expiration date
No expiration date

Use a finite expiration time for app credentials. Use No expiration date only when your security policy permits long-lived credentials, and rotate the credential regularly.

Create an administrator API key

An administrator API key contains the admin access scope. It can perform cluster-wide administrative operations without explicit namespace permissions.

Click New API Key.
Enter a descriptive name.
Select an expiration time.
Turn on Is Admin.
Create an administrator API key with Is Admin turned on.
Click Confirm.
Copy the API key and store it in a secrets manager. Console displays the API key only once.
Click Copy and Close.

Limit administrator API keys to administrative workflows. For apps that only produce or consume messages, create a non-administrator API key and grant only the required permissions.

Create a non-administrator API key

A non-administrator API key contains the access scope. It can’t access Pulsar resources until you grant permissions to its API key ID.

Click New API Key.
Enter a descriptive name.
Select an expiration time.
Turn off Is Admin.
Create a non-administrator API key with Is Admin turned off.
Click Confirm.
Copy the API key and store it in a secrets manager. Console displays the API key only once.
Click Copy and Close.
In the API key list, record the value in the ID column. The ID is the token jti and the Pulsar role that you use to grant permissions.

Never commit an API key to source control. Use a finite expiration time and the least privileges required by the app.

Grant permissions to a non-administrator API key

Use an administrator API key to grant permissions. Set --role to the non-administrator API key’s ID, not its name or API key value:

pulsarctl --admin-service-url http://<Your PulsarProxy Endpoint>:8080 \
  --token "<admin-api-key>" \
  namespaces grant-permission \
  --role <non-admin-api-key-id> \
  --actions produce \
  --actions consume \
  public/default

Expected output:

Grant permissions [produce consume] to the client role <non-admin-api-key-id> to access the namespace public/default successfully

Connect a Pulsar client

The API key service issues API keys as JWT tokens. Configure Pulsar clients to use an API key in the same way that you configure a client for JWT authentication. Use the AuthenticationToken plugin and pass the API key as the token value. You don’t need an API-key-specific client plugin. Produce messages with the non-administrator API key:

bin/pulsar-client \
  --url pulsar://<Your PulsarProxy Endpoint>:6650 \
  --auth-plugin org.apache.pulsar.client.impl.auth.AuthenticationToken \
  --auth-params "token:<non-admin-api-key>" \
  produce public/default/test -m "test" -n 10

Consume messages with the same API key:

bin/pulsar-client \
  --url pulsar://<Your PulsarProxy Endpoint>:6650 \
  --auth-plugin org.apache.pulsar.client.impl.auth.AuthenticationToken \
  --auth-params "token:<non-admin-api-key>" \
  consume public/default/test -s my-subscription -n 10

Both commands complete successfully when the API key role has produce and consume permissions on public/default.

Use the API key value that Console displays after creation for --auth-params. Use the API key ID only as the Pulsar role when granting permissions.

Revoke an API key

In Console, select API Keys.
Open the actions menu for the API key that you want to revoke, and then select Revoke.
Open an API key's actions menu and select Revoke.
Confirm the revocation.

Brokers and proxies cache the revocation list, so they might continue to accept the credential until the next refresh. Allow at least 65 seconds before verifying that Pulsar rejects the revoked credential with 401 Unauthorized.

Troubleshoot API key authentication

The broker rejects the original principal

If a non-administrator client fails with an error similar to cannot specify originalPrincipal when connecting without valid proxy role, compare the proxy credential jti with the broker configuration:

kubectl get secret private-cloud-apikeys-proxy-key -n pulsar \
  -o json | jq -r '.data.jti | @base64d'

kubectl get pulsarbroker private-cloud -n pulsar \
  -o jsonpath='{.spec.config.authorization.proxyRoles}'

If the proxy jti is missing from the broker’s proxy roles, follow Apply the proxy role workaround, and wait for the brokers to become ready.

A revoked API key still works

Wait for the revocation list to refresh, and then open a new client connection. An existing authenticated connection might remain active until it reconnects.

Console can’t create an API key

Confirm that spec.config.server.allowCustomClaims is true and spec.config.server.multiTenant.enable is false or omitted. Console sets the API key scope and sub claims when it creates the credential.

Overview

Quick Start

Configure Private Cloud

Operate Private Cloud

References

Before you begin

Deploy the API key service

Prepare the API key backend topic

Enable automatic topic creation

Create the topic manually

Enable API key authentication on the cluster

Apply the proxy role workaround

Configure Console for API key management

Create API keys in Console

Choose an expiration time

Create an administrator API key

Create a non-administrator API key

Grant permissions to a non-administrator API key

Connect a Pulsar client

Revoke an API key

Troubleshoot API key authentication

The broker rejects the original principal

A revoked API key still works

Console can’t create an API key

​Before you begin

​Deploy the API key service

​Prepare the API key backend topic

​Enable automatic topic creation

​Create the topic manually

​Enable API key authentication on the cluster

​Apply the proxy role workaround

​Configure Console for API key management

​Create API keys in Console

​Choose an expiration time

​Create an administrator API key

​Create a non-administrator API key

​Grant permissions to a non-administrator API key

​Connect a Pulsar client

​Revoke an API key

​Troubleshoot API key authentication

​The broker rejects the original principal

​A revoked API key still works

​Console can’t create an API key

Before you begin

Deploy the API key service

Prepare the API key backend topic

Enable automatic topic creation

Create the topic manually

Enable API key authentication on the cluster

Apply the proxy role workaround

Configure Console for API key management

Create API keys in Console

Choose an expiration time

Create an administrator API key

Create a non-administrator API key

Grant permissions to a non-administrator API key

Connect a Pulsar client

Revoke an API key

Troubleshoot API key authentication

The broker rejects the original principal

A revoked API key still works

Console can’t create an API key