streamnative.io
Help Center
Login
  1. StreamNative CLI
  2. Other Tools

Use kcat With StreamNative Cloud

kcat is a popular CLI tool that you can use to test and debug your StreamNative Cloud clusters using the Kafka protocol. You can use kcat to produce, consume, and list topic and partition information for Kafka topics. Described as "netcat for Kafka", it is a swiss-army knife of tools for inspecting and creating data in Kafka.

It is similar to Kafka Console Producer (kafka-console-producer.sh) and Kafka Console Consumer (kafka-console-consumer.sh), but it offers more features and is easier to use.

Create a configuration file

Any librdkafka configuration property can be set on the command line using -X key=value, or in a configuration file specified by -F <config-file>.

  1. Create a file named cloud.properties and save it in a secure location. You will populate this file with credentials to access your StreamNative Cloud account so you must keep in a safe place. Add the following content to the file:

    Warning

    Due to length limitations, StreamNative's API Key should be specified in the command line using the -X option rather than in the configuration file.

    bootstrap.servers=<broker-endpoint>
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.username=unused

  2. Next, you will populate the file with your StreamNative Cloud cluster information.

    1. Sign in to your StreamNative Cloud account.

    2. In the Cloud Console, navigate to the cluster you want to connect to.

    3. In the Cluster Dashboard page, select the Details tab.

    4. Copy the Kafka Service URL (TCP) value and paste it into the bootstrap.servers property in the cloud.properties file.

    5. Follow the instructions in Create an API key to create an API key and paste the API Key value into the cloud.properties file.

Use the tool

After you have set up the configuration file that references your cluster and noted the API Key, you can use kcat to produce and consume messages. You will also need the bootstrap server of the cluster when you run the tool.

The following examples demonstrate how to use kcat to produce and consume messages.

Produce messages

Run the following command to produce messages to a topic test_kcat. Before running this command:

  1. Replace <bootstrap-server> with your cluster's actual bootstrap server address
  2. Update the path to point to your cloud.properties file location
  3. Replace <api-key> with your actual API key
kcat -b <bootstrap-server> -t test_kcat -F /path/to/secure/location/cloud.properties -X "sasl.password=token:<api-key>" -P -K:

The -K: flag indicates that messages should be formatted as key:value. Enter messages in this format at the prompt:

1:apple
2:orange
3:pear
4:grape

After that, you type Ctrl-D to send the messages to the topic.

Consume messages

Now, you can consume messages from the topic test_kcat using the following command. Before running this command:

  1. Replace <bootstrap-server> with your cluster's actual bootstrap server address
  2. Update the path to point to your cloud.properties file location
  3. Replace <api-key> with your actual API key
kcat -b <bootstrap-server> -t test_kcat -F /path/to/secure/location/cloud.properties -X "sasl.password=token:<api-key>" -C -f 'Key: %k, Value: %s\n'

You should see the following output:

Key: 1, Value: apple
Key: 2, Value: orange
Key: 3, Value: pear
Key: 4, Value: grape
Previous
Use Kafka Tools