Monitor and Troubleshoot Kafka Connectors

StreamNative Cloud allows you to monitor kafka connectors’ status and logs.

View connector status

This section describes how to view connector status using snctl, kcctl, Rest API, and console. The following example introduces how to view the status of the data generator source connector named test.

snctl
kcctl
Rest API
Console

To check the status of the source connector test, run the following command:

snctl kafka admin connect describe connector test

Name:       test
Type:       source
State:      RUNNING
Worker ID:  10.80.161.252:8083
Config:
  config.action.reload:               restart
  connector.class:                    io.confluent.kafka.connect.datagen.DatagenConnector
  errors.log.enable:                  false
  errors.log.include.messages:        false
  errors.retry.delay.max.ms:          60000
  errors.retry.timeout:               0
  errors.tolerance:                   none
  exactly.once.source.support:        disabled
  exactly.once.support:               requested
  header.converter:                   org.apache.kafka.connect.storage.SimpleHeaderConverter
  iterations:                         -1
  kafka.topic:                        testgen
  key.converter:                      org.apache.kafka.connect.json.JsonConverter
  max.interval:                       500
  name:                               test
  offset.storage.partitions:          25
  offset.storage.replication.factor:  3
  offset.storage.topic:               kafka-connect-offset-storage
  quickstart:                         users
  tasks.max:                          1
  tasks.max.enforce:                  true
  transaction.boundary:               poll
  value.converter:                    org.apache.kafka.connect.json.JsonConverter
Tasks:
  0:
    State:      RUNNING
    Worker ID:  10.80.161.252:8083
Topics:
  testgen

To check the status of the source connector test, run the following command:

kcctl describe connectors test

Name:       test
Type:       source
State:      RUNNING
Worker ID:  10.80.161.252:8083
Config:
  config.action.reload:               restart
  connector.class:                    io.confluent.kafka.connect.datagen.DatagenConnector
  errors.log.enable:                  false
  errors.log.include.messages:        false
  errors.retry.delay.max.ms:          60000
  errors.retry.timeout:               0
  errors.tolerance:                   none
  exactly.once.source.support:        disabled
  exactly.once.support:               requested
  header.converter:                   org.apache.kafka.connect.storage.SimpleHeaderConverter
  iterations:                         -1
  kafka.topic:                        testgen
  key.converter:                      org.apache.kafka.connect.json.JsonConverter
  max.interval:                       500
  name:                               test
  offset.storage.partitions:          25
  offset.storage.replication.factor:  3
  offset.storage.topic:               kafka-connect-offset-storage
  quickstart:                         users
  tasks.max:                          1
  tasks.max.enforce:                  true
  transaction.boundary:               poll
  value.converter:                    org.apache.kafka.connect.json.JsonConverter
Tasks:
  0:
    State:      RUNNING
    Worker ID:  10.80.161.252:8083
Topics:
  testgen

To check the status of the source connector test, run the following command:

curl -X GET "https://public%2Fdefault:${APIKEY}@${KAFKA-SERVICE-URL}/admin/kafkaconnect/connectors/test"

You should see the following output:

{
  "name": "test",
  "config": {
    "config.action.reload": "restart",
    "connector.class": "io.confluent.kafka.connect.datagen.DatagenConnector",
    "errors.log.enable": "false",
    "errors.log.include.messages": "false",
    "errors.retry.delay.max.ms": "60000",
    "errors.retry.timeout": "0",
    "errors.tolerance": "none",
    "exactly.once.source.support": "disabled",
    "exactly.once.support": "requested",
    "header.converter": "org.apache.kafka.connect.storage.SimpleHeaderConverter",
    "iterations": "-1",
    "kafka.topic": "testgen",
    "key.converter": "org.apache.kafka.connect.json.JsonConverter",
    "max.interval": "500",
    "name": "test",
    "offset.storage.partitions": "25",
    "offset.storage.replication.factor": "3",
    "offset.storage.topic": "kafka-connect-offset-storage",
    "quickstart": "users",
    "tasks.max": "1",
    "tasks.max.enforce": "true",
    "transaction.boundary": "poll",
    "value.converter": "org.apache.kafka.connect.json.JsonConverter"
  },
  "tasks": [
    {
      "connector": "test",
      "task": 0
    }
  ],
  "type": "source"
}

View connector logs

View the connector logs using `snctl`

This section describes how to view connector logs using snctl.

This example assumes you have installed snctl and initialized snctl configurations.

You can run the snctl logs command to view logs for a specific connector. This table outlines the configuration options that are used for viewing connector logs. For details about all supported fields, you can use the snctl logs -h command to list more information.

Option	Descriptions
`-c` or `--cluster`	The name of your Pulsar cluster where the connector is created.
`-p` or `--component`	The type of component to monitor. Available options are `function`, `sink`, `source` and `kafka-connect`.
`-f` or `--follow`	Continuously list the connector log history.
`-h` or `--help`	Show usage information about the `snctl logs` command.
`-i` or `--instance`	The name of your Pulsar instance where the connector is created.
`--name`	The name of your connector.
`-o` or `--organization`	The name of your organization where the connector is created.
`--previous`	Print the logs that are generated before the configured timestamp.
`--pulsar-tenant`	The name of your Pulsar tenant where the connector is created.
`--pulsar-namespace`	The name of your Pulsar namespace where the connector is created.
`--since`	List logs more recent than the specific time. Available units are `second`, `minute`, and `hour`, such as `24h`.
`-s` or `--size`	Specify how many lines of recent logs to display.
`--timestamp`	Include timestamps on each line in the log output.

The following command example shows how to view up to 60 lines of the data1 sink connector’s logs within the last 5 hours.

snctl logs --since 5h --organization sndev --instance aws --cluster aws --name data1 --pulsar-tenant public --pulsar-namespace default -p kafka-connect -f -s 60

You should see the following output:

2024-08-26T04:37:21,290+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 241 acknowledged messages
2024-08-26T04:38:21,306+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 229 acknowledged messages
2024-08-26T04:39:13,572+0000 [kafka-admin-client-thread | datagen-ui-73c8b112-mesh-admin] INFO  org.apache.kafka.clients.NetworkClient - [AdminClient clientId=datagen-ui-73c8b112-mesh-admin] Node 1753251735 disconnected.
2024-08-26T04:39:21,320+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 233 acknowledged messages
2024-08-26T04:40:18,606+0000 [kafka-producer-network-thread | connector-producer-datagen-ui-0] INFO  org.apache.kafka.clients.NetworkClient - [Producer clientId=connector-producer-datagen-ui-0] Node 1261922880 disconnected.
2024-08-26T04:40:21,334+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 261 acknowledged messages
2024-08-26T04:41:21,349+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 235 acknowledged messages
2024-08-26T04:42:21,362+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 270 acknowledged messages
2024-08-26T04:43:21,381+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 235 acknowledged messages
2024-08-26T04:44:13,738+0000 [kafka-admin-client-thread | datagen-ui-73c8b112-mesh-admin] INFO  org.apache.kafka.clients.NetworkClient - [AdminClient clientId=datagen-ui-73c8b112-mesh-admin] Node 1261922880 disconnected.
2024-08-26T04:44:21,393+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 240 acknowledged messages
2024-08-26T04:45:21,407+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 227 acknowledged messages
2024-08-26T04:46:21,421+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 234 acknowledged messages
2024-08-26T04:47:21,436+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 226 acknowledged messages
2024-08-26T04:48:21,449+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 248 acknowledged messages
2024-08-26T04:49:13,883+0000 [kafka-admin-client-thread | datagen-ui-73c8b112-mesh-admin] INFO  org.apache.kafka.clients.NetworkClient - [AdminClient clientId=datagen-ui-73c8b112-mesh-admin] Node 1753251735 disconnected.
2024-08-26T04:49:21,462+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 231 acknowledged messages
2024-08-26T04:50:21,479+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 218 acknowledged messages
2024-08-26T04:51:21,492+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 241 acknowledged messages
2024-08-26T04:52:21,509+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 247 acknowledged messages
2024-08-26T04:53:21,530+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 230 acknowledged messages
2024-08-26T04:54:14,033+0000 [kafka-admin-client-thread | datagen-ui-73c8b112-mesh-admin] INFO  org.apache.kafka.clients.NetworkClient - [AdminClient clientId=datagen-ui-73c8b112-mesh-admin] Node 1261922880 disconnected.
2024-08-26T04:54:18,658+0000 [kafka-producer-network-thread | connector-producer-datagen-ui-0] INFO  org.apache.kafka.clients.NetworkClient - [Producer clientId=connector-producer-datagen-ui-0] Node 1261922880 disconnected.
2024-08-26T04:54:21,559+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 221 acknowledged messages
2024-08-26T04:55:21,585+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 238 acknowledged messages
2024-08-26T04:56:21,614+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 224 acknowledged messages
2024-08-26T04:57:21,643+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 226 acknowledged messages
2024-08-26T04:58:21,655+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 236 acknowledged messages
2024-08-26T04:59:14,185+0000 [kafka-admin-client-thread | datagen-ui-73c8b112-mesh-admin] INFO  org.apache.kafka.clients.NetworkClient - [AdminClient clientId=datagen-ui-73c8b112-mesh-admin] Node 1753251735 disconnected.
2024-08-26T04:59:21,673+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 242 acknowledged messages
2024-08-26T05:00:21,690+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 233 acknowledged messages
2024-08-26T05:01:21,715+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 244 acknowledged messages
2024-08-26T05:02:21,728+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 242 acknowledged messages
2024-08-26T05:03:21,743+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 249 acknowledged messages
2024-08-26T05:04:14,338+0000 [kafka-admin-client-thread | datagen-ui-73c8b112-mesh-admin] INFO  org.apache.kafka.clients.NetworkClient - [AdminClient clientId=datagen-ui-73c8b112-mesh-admin] Node 1261922880 disconnected.
2024-08-26T05:04:21,758+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 226 acknowledged messages
2024-08-26T05:05:21,779+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 249 acknowledged messages
2024-08-26T05:06:21,800+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 234 acknowledged messages
2024-08-26T05:07:21,812+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 249 acknowledged messages
2024-08-26T05:08:21,827+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 239 acknowledged messages
2024-08-26T05:09:14,476+0000 [kafka-admin-client-thread | datagen-ui-73c8b112-mesh-admin] INFO  org.apache.kafka.clients.NetworkClient - [AdminClient clientId=datagen-ui-73c8b112-mesh-admin] Node 1753251735 disconnected.
2024-08-26T05:09:21,841+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 232 acknowledged messages
2024-08-26T05:10:21,855+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 252 acknowledged messages
2024-08-26T05:11:21,870+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 242 acknowledged messages
2024-08-26T05:12:21,884+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 234 acknowledged messages
2024-08-26T05:13:21,897+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 237 acknowledged messages
2024-08-26T05:14:14,648+0000 [kafka-admin-client-thread | datagen-ui-73c8b112-mesh-admin] INFO  org.apache.kafka.clients.NetworkClient - [AdminClient clientId=datagen-ui-73c8b112-mesh-admin] Node 1261922880 disconnected.
2024-08-26T05:14:21,909+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 237 acknowledged messages
2024-08-26T05:15:21,920+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 234 acknowledged messages
2024-08-26T05:16:21,934+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 248 acknowledged messages
2024-08-26T05:17:21,953+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 232 acknowledged messages
2024-08-26T05:18:21,972+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 238 acknowledged messages
2024-08-26T05:19:14,802+0000 [kafka-admin-client-thread | datagen-ui-73c8b112-mesh-admin] INFO  org.apache.kafka.clients.NetworkClient - [AdminClient clientId=datagen-ui-73c8b112-mesh-admin] Node 1753251735 disconnected.
2024-08-26T05:19:21,991+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 226 acknowledged messages
2024-08-26T05:20:22,004+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 235 acknowledged messages
2024-08-26T05:21:22,018+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 251 acknowledged messages
2024-08-26T05:22:22,033+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 238 acknowledged messages
2024-08-26T05:23:22,047+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 227 acknowledged messages
2024-08-26T05:24:14,938+0000 [kafka-admin-client-thread | datagen-ui-73c8b112-mesh-admin] INFO  org.apache.kafka.clients.NetworkClient - [AdminClient clientId=datagen-ui-73c8b112-mesh-admin] Node 1261922880 disconnected.
2024-08-26T05:24:22,061+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 236 acknowledged messages

View the connector logs from log topics

StreamNative Cloud provide a way to send your logs to a Pulsar topic, you can just use config sn.log.topic to set it. Once the connector is deployed with this config, you can configure consumers to consume messages from the log topic.

What’s next?

SMTs(Single Message Transformations)
Discover kafka Connect Ecosystem on StreamNative Hub.

Introduction

Get Started

Clusters

Data Streams

Security

Governance

Connect

Lakehouse

Agents (Orca Engine)

Process

Networking

Log And Monitor

Universal Linking

Billing

References

View connector status

View connector logs

View the connector logs using `snctl`

View the connector logs from log topics

What’s next?

Introduction

Get Started

Clusters

Data Streams

Security

Governance

Connect

Lakehouse

Agents (Orca Engine)

Process

Networking

Log And Monitor

Universal Linking

Billing

References

​View connector status

​View connector logs

​View the connector logs using snctl

​View the connector logs from log topics

​What’s next?

View connector status

View connector logs

View the connector logs using `snctl`

View the connector logs from log topics

What’s next?