Prerequisites
- If you don’t already have a collection in Zilliz Cloud or Milvus cluster, create a collection with a vector field.
- Collect the
endpoint
,token
, andcollection.name
parameters from your Zilliz Cloud instance or Milvus cluster.
Quick Start
- Setup the kcctl client: doc
-
Set up a Milvus cluster, we can set up a local Milvus cluster using docker-compose:
-
Initialize the local Milvus cluster, below is a python script to do so:
-
Create a JSON file like the following:
-
Run the following command to create the connector:
Configuration
The Milvus Kafka Connect Sink connector is configured using the following Required properties:Parameter | Description |
---|---|
public.endpoint | The endpoint of your Zilliz Cloud instance or Milvus cluster. |
token | The token of your Zilliz Cloud instance or Milvus cluster. |
database.name | The name of the database to write to. |
collection.name | The name of the collection to write to. |
topics | The Kafka topics to read from. |
Known Issues
- The Milvus Kafka Connect Sink connector does not support the dynamic fields feature yet, and when the dynamic fields feature enabled on the Milvus server, or enabled on the Zilliz Cloud instance, the connector will not work properly and throw below error: