Kafka Connect JR - StreamNative Documentation

kafka-connect-jr-source is a Kafka Connect connector for generating mock data for testing and is not suitable for production scenarios. It is available in the StreamNative Cloud.

This connector is available as a built-in connector on StreamNative Cloud.

Quick Start

Setup the kcctl client: doc

Create a json file like below:

{
    "name": "jr-quickstart",
    "config": {
        "connector.class": "io.jrnd.kafka.connect.connector.JRSourceConnector",
        "template": "net_device",
        "topic": "net_device",
        "frequency": 5000,
        "objects": 5,
        "tasks.max": 1
    }
}

Run the following command to create the connector:
kcctl apply -f <filename>.json

Configuration

The JR Source Connector can be configured using the following properties:

Parameter	Description	Default
`template`	A valid JR existing template name. Skipped when `embedded_template` is set.	net_device
`topic`	Destination topic on Kafka
`frequency`	Repeat the creation of a random object every ‘frequency’ milliseconds.	5000
`duration`	Set a time bound to the entire object creation. The duration is calculated starting from the first run and is expressed in milliseconds. At least one run will always been scheduled, regardless of the value for ‘duration’. If not set creation will run forever.	-1
`objects`	Number of objects to create at every run.	1
`key_field_name`	Name for key field, for example ‘ID’. This is an OPTIONAL config, if not set, objects will be created without a key. Skipped when `key_embedded_template` is set. Value for key will be calculated using JR function `key`.
`key_value_interval_max`	Maximum interval value for key value, for example 150 (0 to key_value_interval_max). Skipped when `key_embedded_template` is set.	100
`value.converter`	One between `org.apache.kafka.connect.storage.StringConverter`, `io.confluent.connect.avro.AvroConverter`, `io.confluent.connect.json.JsonSchemaConverter` or `io.confluent.connect.protobuf.ProtobufConverter`	org.apache.kafka.connect.storage.StringConverter
`value.converter.schema.registry.url`	Only if `value.converter` is set to `io.confluent.connect.avro.AvroConverter`, `io.confluent.connect.json.JsonSchemaConverter` or `io.confluent.connect.protobuf.ProtobufConverter`. URL for Schema Registry.
`key.converter`	One between `org.apache.kafka.connect.storage.StringConverter`, `io.confluent.connect.avro.AvroConverter`, `io.confluent.connect.json.JsonSchemaConverter` or `io.confluent.connect.protobuf.ProtobufConverter`	org.apache.kafka.connect.storage.StringConverter
`key.converter.schema.registry.url`	Only if `key.converter` is set to `io.confluent.connect.avro.AvroConverter`, `io.confluent.connect.json.JsonSchemaConverter` or `io.confluent.connect.protobuf.ProtobufConverter`. URL for Schema Registry.

Available templates on StreamNative Cloud

csv_product
csv_user
finance_stock_trade
fleet_mgmt_sensors
fleetmgmt_description
fleetmgmt_location
fleetmgmt_sensor
gaming_game
gaming_player
gaming_player_activity
insurance_customer
insurance_customer_activity
insurance_offer
inventorymgmt_inventory
inventorymgmt_product
iot_device_information
marketing_campaign_finance
net_device
payment_credit_card
payment_transaction
payroll_bonus
payroll_employee
payroll_employee_location
pizzastore_order
pizzastore_order_cancelled
pizzastore_order_completed
pizzastore_util
shoestore_clickstream
shoestore_customer
shoestore_order
shoestore_shoe
shopping_order
shopping_purchase
shopping_rating
siem_log
store
syslog_log
user
user_with_key
users
users_array_map
util_ip
util_userid
webanalytics_clickstream
webanalytics_code
webanalytics_page_view
webanalytics_user

Sources

Sinks

Kafka connect jr source

Quick Start

Configuration

Available templates on StreamNative Cloud

​Quick Start

​Configuration

​Available templates on StreamNative Cloud

Quick Start

Configuration

Available templates on StreamNative Cloud