CPU and Memory Resources

Before configuring CPU and memory resource requirements, you need to review the cluster size for planning resource allocation. It is important to set proper requests and limits for the performance of the StreamNative Platform clusters and their applications. For more information about CPU and memory resources in Kubernetes, see resource management for Pods and Containers.

StreamNative Platform allows you to define custom Pod resource requirements for the components that it deploys. You can specify these requirements using the requests and limits properties for components in the values.yaml YAML file.

<component>: # --- [1]
  resources: # --- [2]
    limits: # --- [3]
      cpu: # --- [4]
      memory: # --- [5]
    requests: # --- [6]
      cpu: # --- [7]
      memory: # --- [8]

• [1] <component>: represent the component to configure (i.e. broker, proxy, etc).

• [2] resources: describe the compute resource requirements for this component.

• [3] limits: describe the maximum amount of allowed compute resources. This component will throttle if it tries to use more resources than the configured value.

• [5] requests: describe the minimum amount of required compute resources. If the requests section is omitted, it takes limits values by default if that are explicitly specified in the same component. Otherwise, the request CPU and memory are set to the values defined for the Kubernetes cluster.

• [4][7] cpu: CPU resources in the limits and requests sections are measured in CPU units. One CPU unit is equivalent to one physical CPU core or one virtual CPU core.

  Fractional CPU requests are also allowed. For example, resources.requests.cpu: 0.5 requests half as much CPU time.

  For CPU resource units, the quantity expression 0.1 is equivalent to the expression 100m, which means "one hundred milli CPU".

• [5][8] memory: Memories in the limits and requests sections are measured in bytes.

The following example specifies a request for 0.25 CPU and 64 MiB of memory. Its limits are set to 0.5 CPU and 128 MiB of memory.

broker:
  resources:
    limits:
      cpu: '500m'
      memory: '128Mi'
    requests:
      cpu: '250m'
      memory: '64Mi'

Define Java heap size

In addition to setting memory sizing, you need to define the Java (JVM) heap size.

You can define the Java heap size for StreamNative Platform components using the Configuration overrides feature (<component>.jvm.memoryOptions) in the component section.

Here's an example where the JVM heap size is set to 6 GB:

broker:
  jvm:
    memoryOptions:
      - >
        -Xms6g -Xmx6g

For more details about the JVM options, see configure JVM options.