Advanced Configuration

StreamNative Platform provides many default configurations to make it easier for deployment. Sometimes, you may want to customize the StreamNative Platform based on your actual situation. Therefore, StreamNative Platform provides a few advanced configuration options, including:

Override configurations

You can override default configuration parameters of a StreamNative Platform component in the component section of the values.yaml YAML file as below:

<component>:
  configData: {}

Refer to the configuration reference for configuration parameters used in StreamNative Platform components. For configuration options that do not exist in the default configuration files, you can add prefix PULSAR_PREFIX_ to those parameters.

The example below shows how to disable automatic bundle split (By default, it is enabled.) for the Pulsar broker:

broker:
  configData:
    loadBalancerAutoBundleSplitEnabled: 'false'

Configure JVM options

You can set the following Java Virtual Machine (JVM) options on Pulsar core components. These options are used for tuning the JVM to improve performance or reduce high memory consumption.

memoryOptions: describe the options that are used to define the Java heap size for StreamNative Platform components.
gcOptions: describe the options that are used to define Garbage Collector (GC).
extraOptions: describe the additional options that are used to define the JVM. It takes a list of strings.

`memoryOptions`

-Xms: define the starting heap size for JVM. For example, Xms2g means that the initial Java heap size of JVM is 2 GB.
-Xmx: define the maximum heap size of JVM. For example, Xmx6g means that the maximum Java heap size is 6 GB.
XX:MaxDirectMemorySize: define the maximum direct buffer size. For example, XX:MaxDirectMemorySize=2048m means that the maximum direct buffer size is 2 GB.

In general, you should always set -Xms and -Xmx together.

The following example shows how to define 2 GB direct buffer size in the Pulsar broker Pod.

broker:
  jvm:
    memoryOptions:
      - >
        XX:MaxDirectMemorySize=2048m

`gcOptions`

For configuration options that do not exist in the default configuration file (values.yaml), you can add them as required. For details about JVM options, see Java HotSpot VM Options.

-XX:+UseG1GC: use the Garbage First Garbage Collector (G1 GC) as the GC.
-XX:MaxGCPauseMillis: set the peak pause time expected in the environment.
-Dio.netty.leakDetectionLevel: set the leak detection level.
-Dio.netty.recycler.linkCapacity: set the capacity of the recycler.
-Dcom.sun.management.jmxremote: enable remote JMX access.
-Djute.maxbuffer: set the file size limit for the ZooKeeper cluster.
-XX:+ParallelRefProcEnabled: enable the parallel reference processing.
-XX:+UnlockExperimentalVMOptions: unlock the options that provide experimental features in the JVM.
-XX:+AggressiveOpts: enable the use of aggressive performance optimization features.
-XX:+DoEscapeAnalysis: enable the use of escape analysis.
-XX:ParallelGCThreads: set the number of GC threads.
-XX:ConcGCThreads: set the number of threads that concurrent garbage collectors will use.
-XX:G1NewSizePercent: set the percentage (0-100) of the heap size to use as default minimum Young gen size.
-XX:+DisableExplicitGC: disable the explicit GC.
-XX:-ResizePLAB: disable resizing PPromotion Local Allocation Buffers (PLABs).
-XX:+ExitOnOutOfMemoryError: trigger a shut down on VM out-of-memory conditions.
-XX:+PerfDisableSharedMem: disable to create a process log directory under /tmp/hperf[username].
-Dzookeeper.forceSync: set whether to synchronize updates to the media of the transaction log before finishing processing the update.
-verbosegc: configure the Verbose Garbage Collection (VGC).
-Xlog:gc:/var/log/bookie-gc.log: configure the GC logging file for the bookie.

The following example shows how to define the G1 GC in the Pulsar broker Pod.

broker:
  jvm:
    gcOptions:
      - >
        -XX:+UseG1GC

Define Pod annotations

An annotation in Kubernetes defines an unstructured Key Value Map (KVM) that can be set by external tools to store and retrieve metadata. Annotations are stored with Pods. annotations must be a map of string keys and string values. You can define custom annotations for StreamNative Platform components, and those annotations are applied to the Kubernetes Pods for the components.

Annotation values must pass Kubernetes annotations validation. See the Kubernetes documentation on Annotations for details.

Configure annotations of the StreamNative components in the values.yaml YAML file as follows, and update the resource.

<component>:
  annotations:
    key1: value1
    key2: value2

The following are example annotations in the Pulsar broker Pod for the Hashicorp vault:

broker:
  annotations:
    vault.hashicorp.com/agent-inject: true
    vault.hashicorp.com/agent-inject-status: update
    vault.hashicorp.com/preserve-secret-case: true
    vault.hashicorp.com/agent-inject-secret-jksPassword.txt: secret/jksPassword.txt

Define service annotations

Similar to defining Pod annotations, you can also add annotations to a service object for each StreamNative Platform component. annotations must be a map of string keys and string values.

Configure annotations in a service of the StreamNative components in the values.yaml YAML file as follows, and update the resource.

<component>:
  service:
    annotations:
      key1: value1
      key2: value2

The following are example annotations in the Pulsar Proxy service for creating a Network Load Balancer (NLB) with a certificate [ACM Certificate ARN]:

proxy:
  service:
    annotations:
      service.beta.kubernetes.io/aws-load-balancer-type: nlb
      service.beta.kubernetes.io/aws-load-balancer-ssl-cert: '[ACM Certificate ARN]'
      service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: 'true'

Define Pod labels

In Kubernetes, a label defines an unstructured Key Value Map (KVM) to specify attributes of objects. You can use labels to organize and to select subsets of objects. Labels are stored with Pods. labels must be a map of string keys and string values. You can define custom labels for StreamNative Platform components, and those labels are applied to the Kubernetes Pods for the components.

Label values must pass Kubernetes label validation. See the Kubernetes documentation on labels for details.

Configure labels of the StreamNative components in the values.yaml YAML file as follows, and update the resource.

<component>:
  labels:
    key1: value1
    key2: value2

Define extra environment variables

In Kubernetes, when you create a Pod (with a Deployment, StatefulSet, or other means), you can set environment variables for the containers that run in the Pod. Then, Kubernetes will pass these environment variables to the application(s) inside the Pods. StreamNative Platform provides the extraEnv option, which takes a list of maps. You can define custom environment variables for StreamNative Platform components, and those environment variables are applied to the Kubernetes Pods for the components.

Configure environment variables of the StreamNative components in the values.yaml YAML file as follows, and update the resource.

<component>:
  extraEnv:
    - name: name1
      value: value1
    - name: name2
      value: value2

Mount additional Secrets

A Secret in Kubernetes is an object that holds some confidential data such as a password, a token, or a key. StreamNative Platform provides the extraSecretRefs option, which is used to mount secrets used by a container in a Pod. You can define Secrets for StreamNative Platform components, and those Secrets are applied to the Kubernetes Pods for the components.

Mount Secrets of the StreamNative components in the values.yaml YAML file as follows, and update the resource.

<component>:
  extraSecretRefs:
    - mountPath: /path/to/mount # --- [1]
      secretName: '[secret name]' # --- [2]

[1] mountPath: the path to mount the Secret on.
[2] secretName: the name of the Secret to mount.

Disable init containers

For StreamNative Platform, init containers are used to set some parameters of TCP connection. These parameters are used to tune TCP keepalive for Pulsar broker and autorecovery pods. But the init container for Pulsar brokers and autorecovery requires superuser (runAsRoot) privileged permissions because it needs to execute sysctl. The init container is entirely optional if you take care of setting TCP keepalive entirely outside of Kubernetes and on the host. In addition, in many OpenShift environments, the privilege escalation is disabled. Without privilege escalation, the init container will fail and the deployment will stop. In these cases, you can disable the init container that is used to run sysctl.

By default, the init container is enabled. To disable the init container, you can set the cloud.streamnative.io/omit-init-sysctl annotation to true on PulsarBroker and BookKeeperCluster CRs, as shown below:

broker:
  pulsarBroker:
    annotations:
      cloud.streamnative.io/omit-init-sysctl: 'true'

bookkeeper:
  bookKeeperCluster:
    annotations:
      cloud.streamnative.io/omit-init-sysctl: 'true'