StreamNative Cloud allows you to monitor kafka connectors’ status and logs.

​
View connector status

This section describes how to view connector status using snctl, kcctl, Rest API, and console. The following example introduces how to view the status of the data generator source connector named test.
To check the status of the source connector test, run the following command:
snctl kafka admin connect describe connector test

Name:       test
Type:       source
State:      RUNNING
Worker ID:  10.80.161.252:8083
Config:
  config.action.reload:               restart
  connector.class:                    io.confluent.kafka.connect.datagen.DatagenConnector
  errors.log.enable:                  false
  errors.log.include.messages:        false
  errors.retry.delay.max.ms:          60000
  errors.retry.timeout:               0
  errors.tolerance:                   none
  exactly.once.source.support:        disabled
  exactly.once.support:               requested
  header.converter:                   org.apache.kafka.connect.storage.SimpleHeaderConverter
  iterations:                         -1
  kafka.topic:                        testgen
  key.converter:                      org.apache.kafka.connect.json.JsonConverter
  max.interval:                       500
  name:                               test
  offset.storage.partitions:          25
  offset.storage.replication.factor:  3
  offset.storage.topic:               kafka-connect-offset-storage
  quickstart:                         users
  tasks.max:                          1
  tasks.max.enforce:                  true
  transaction.boundary:               poll
  value.converter:                    org.apache.kafka.connect.json.JsonConverter
Tasks:
  0:
    State:      RUNNING
    Worker ID:  10.80.161.252:8083
Topics:
  testgen

​
View connector logs

​
View the connector logs using snctl

This section describes how to view connector logs using snctl.
This example assumes you have installed snctl and initialized snctl configurations.
You can run the snctl logs command to view logs for a specific connector. This table outlines the configuration options that are used for viewing connector logs. For details about all supported fields, you can use the snctl logs -h command to list more information.
OptionDescriptions
-c or --clusterThe name of your Pulsar cluster where the connector is created.
-p or --componentThe type of component to monitor. Available options are function, sink, source and kafka-connect.
-f or --followContinuously list the connector log history.
-h or --helpShow usage information about the snctl logs command.
-i or --instanceThe name of your Pulsar instance where the connector is created.
--nameThe name of your connector.
-o or --organizationThe name of your organization where the connector is created.
--previousPrint the logs that are generated before the configured timestamp.
--pulsar-tenantThe name of your Pulsar tenant where the connector is created.
--pulsar-namespaceThe name of your Pulsar namespace where the connector is created.
--sinceList logs more recent than the specific time. Available units are second, minute, and hour, such as 24h.
-s or --sizeSpecify how many lines of recent logs to display.
--timestampInclude timestamps on each line in the log output.
The following command example shows how to view up to 60 lines of the data1 sink connector’s logs within the last 5 hours. 
snctl logs --since 5h --organization sndev --instance aws --cluster aws --name data1 --pulsar-tenant public --pulsar-namespace default -p kafka-connect -f -s 60
You should see the following output: 
2024-08-26T04:37:21,290+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 241 acknowledged messages
2024-08-26T04:38:21,306+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 229 acknowledged messages
2024-08-26T04:39:13,572+0000 [kafka-admin-client-thread | datagen-ui-73c8b112-mesh-admin] INFO  org.apache.kafka.clients.NetworkClient - [AdminClient clientId=datagen-ui-73c8b112-mesh-admin] Node 1753251735 disconnected.
2024-08-26T04:39:21,320+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 233 acknowledged messages
2024-08-26T04:40:18,606+0000 [kafka-producer-network-thread | connector-producer-datagen-ui-0] INFO  org.apache.kafka.clients.NetworkClient - [Producer clientId=connector-producer-datagen-ui-0] Node 1261922880 disconnected.
2024-08-26T04:40:21,334+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 261 acknowledged messages
2024-08-26T04:41:21,349+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 235 acknowledged messages
2024-08-26T04:42:21,362+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 270 acknowledged messages
2024-08-26T04:43:21,381+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 235 acknowledged messages
2024-08-26T04:44:13,738+0000 [kafka-admin-client-thread | datagen-ui-73c8b112-mesh-admin] INFO  org.apache.kafka.clients.NetworkClient - [AdminClient clientId=datagen-ui-73c8b112-mesh-admin] Node 1261922880 disconnected.
2024-08-26T04:44:21,393+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 240 acknowledged messages
2024-08-26T04:45:21,407+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 227 acknowledged messages
2024-08-26T04:46:21,421+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 234 acknowledged messages
2024-08-26T04:47:21,436+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 226 acknowledged messages
2024-08-26T04:48:21,449+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 248 acknowledged messages
2024-08-26T04:49:13,883+0000 [kafka-admin-client-thread | datagen-ui-73c8b112-mesh-admin] INFO  org.apache.kafka.clients.NetworkClient - [AdminClient clientId=datagen-ui-73c8b112-mesh-admin] Node 1753251735 disconnected.
2024-08-26T04:49:21,462+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 231 acknowledged messages
2024-08-26T04:50:21,479+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 218 acknowledged messages
2024-08-26T04:51:21,492+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 241 acknowledged messages
2024-08-26T04:52:21,509+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 247 acknowledged messages
2024-08-26T04:53:21,530+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 230 acknowledged messages
2024-08-26T04:54:14,033+0000 [kafka-admin-client-thread | datagen-ui-73c8b112-mesh-admin] INFO  org.apache.kafka.clients.NetworkClient - [AdminClient clientId=datagen-ui-73c8b112-mesh-admin] Node 1261922880 disconnected.
2024-08-26T04:54:18,658+0000 [kafka-producer-network-thread | connector-producer-datagen-ui-0] INFO  org.apache.kafka.clients.NetworkClient - [Producer clientId=connector-producer-datagen-ui-0] Node 1261922880 disconnected.
2024-08-26T04:54:21,559+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 221 acknowledged messages
2024-08-26T04:55:21,585+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 238 acknowledged messages
2024-08-26T04:56:21,614+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 224 acknowledged messages
2024-08-26T04:57:21,643+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 226 acknowledged messages
2024-08-26T04:58:21,655+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 236 acknowledged messages
2024-08-26T04:59:14,185+0000 [kafka-admin-client-thread | datagen-ui-73c8b112-mesh-admin] INFO  org.apache.kafka.clients.NetworkClient - [AdminClient clientId=datagen-ui-73c8b112-mesh-admin] Node 1753251735 disconnected.
2024-08-26T04:59:21,673+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 242 acknowledged messages
2024-08-26T05:00:21,690+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 233 acknowledged messages
2024-08-26T05:01:21,715+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 244 acknowledged messages
2024-08-26T05:02:21,728+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 242 acknowledged messages
2024-08-26T05:03:21,743+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 249 acknowledged messages
2024-08-26T05:04:14,338+0000 [kafka-admin-client-thread | datagen-ui-73c8b112-mesh-admin] INFO  org.apache.kafka.clients.NetworkClient - [AdminClient clientId=datagen-ui-73c8b112-mesh-admin] Node 1261922880 disconnected.
2024-08-26T05:04:21,758+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 226 acknowledged messages
2024-08-26T05:05:21,779+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 249 acknowledged messages
2024-08-26T05:06:21,800+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 234 acknowledged messages
2024-08-26T05:07:21,812+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 249 acknowledged messages
2024-08-26T05:08:21,827+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 239 acknowledged messages
2024-08-26T05:09:14,476+0000 [kafka-admin-client-thread | datagen-ui-73c8b112-mesh-admin] INFO  org.apache.kafka.clients.NetworkClient - [AdminClient clientId=datagen-ui-73c8b112-mesh-admin] Node 1753251735 disconnected.
2024-08-26T05:09:21,841+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 232 acknowledged messages
2024-08-26T05:10:21,855+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 252 acknowledged messages
2024-08-26T05:11:21,870+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 242 acknowledged messages
2024-08-26T05:12:21,884+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 234 acknowledged messages
2024-08-26T05:13:21,897+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 237 acknowledged messages
2024-08-26T05:14:14,648+0000 [kafka-admin-client-thread | datagen-ui-73c8b112-mesh-admin] INFO  org.apache.kafka.clients.NetworkClient - [AdminClient clientId=datagen-ui-73c8b112-mesh-admin] Node 1261922880 disconnected.
2024-08-26T05:14:21,909+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 237 acknowledged messages
2024-08-26T05:15:21,920+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 234 acknowledged messages
2024-08-26T05:16:21,934+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 248 acknowledged messages
2024-08-26T05:17:21,953+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 232 acknowledged messages
2024-08-26T05:18:21,972+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 238 acknowledged messages
2024-08-26T05:19:14,802+0000 [kafka-admin-client-thread | datagen-ui-73c8b112-mesh-admin] INFO  org.apache.kafka.clients.NetworkClient - [AdminClient clientId=datagen-ui-73c8b112-mesh-admin] Node 1753251735 disconnected.
2024-08-26T05:19:21,991+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 226 acknowledged messages
2024-08-26T05:20:22,004+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 235 acknowledged messages
2024-08-26T05:21:22,018+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 251 acknowledged messages
2024-08-26T05:22:22,033+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 238 acknowledged messages
2024-08-26T05:23:22,047+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 227 acknowledged messages
2024-08-26T05:24:14,938+0000 [kafka-admin-client-thread | datagen-ui-73c8b112-mesh-admin] INFO  org.apache.kafka.clients.NetworkClient - [AdminClient clientId=datagen-ui-73c8b112-mesh-admin] Node 1261922880 disconnected.
2024-08-26T05:24:22,061+0000 [SourceTaskOffsetCommitter-1] INFO  org.apache.kafka.connect.runtime.WorkerSourceTask - WorkerSourceTask{id=datagen-ui-0} Committing offsets for 236 acknowledged messages

​
View the connector logs from log topics

StreamNative Cloud provide a way to send your logs to a Pulsar topic, you can just use config sn.log.topic to set it. Once the connector is deployed with this config, you can configure consumers to consume messages from the log topic.

​
What’s next?