Develop
We provide a GO SDK for developing Golang Pulsar Functions. The following examples use Pulsar Functions SDK for the Golang language.Feature Matrix
The StreamNative’s Golang runtime doesn’t support full features comparing to Java runtime, and it’s still in developing, below is the matrix:Input Arguments
Input | Java | Go(Pulsar) | Python | Go(StreamNative) |
---|---|---|---|---|
Custom SerDe | ✅ | ❌ | ✅ | ? |
Schema - Avro | ✅ | ❌ | ✅ | ? |
Schema - JSON | ✅ | ❌ | ✅ | ? |
Schema - Protobuf | ✅ | ❌ | ❌ | ? |
Schema - KeyValue | ✅ | ❌ | ❌ | ? |
Schema - AutoSchema | ✅ | ❌ | ❌ | ? |
Scehma - Protobuf Native | ✅ | ❌ | ❌ | ? |
e-2-e encryption | ✅ | ❌ | ✅ | ✅ |
maxMessageRetries | ✅ | ❌ | ❌ | ✅ |
dead-letter policy | ✅ | ❌ | ❌ | ✅ |
SubscriptionName | ✅ | ✅ | ✅ | ✅ |
SubscriptionType | ✅ | ✅ | ✅ | ✅ |
SubscriptionInitialPosition | ✅ | ❌ | ✅ | ✅ |
AutoAck | ✅ | ✅ | ✅ | ✅ |
Users can implement the Schema themselves since we are passing and expecting []byte to/from the users’ function, so leave ? here.
Output Arguments
Output | Java | Go(Pulsar) | Python | Go(StreamNative) |
---|---|---|---|---|
Custom SerDe | ✅ | ❌ | ✅ | ? |
Schema - Avro | ✅ | ❌ | ✅ | ? |
Schema - JSON | ✅ | ❌ | ✅ | ? |
Schema - Protobuf | ✅ | ❌ | ❌ | ? |
Schema - KeyValue | ✅ | ❌ | ❌ | ? |
Schema - AutoSchema | ✅ | ❌ | ❌ | ? |
Schema - Protobuf Native | ✅ | ❌ | ❌ | ? |
useThreadLocalProducers | ✅ | ❌ | ❌ | ✅ |
Key-based Batcher | ✅ | ✅ | ✅ | ✅ |
e-2-e encryption | ✅ | ❌ | ✅ | ✅ |
Compression | ✅ | ✅ | ✅ | ✅ |
Context
Context | Java | Go(Pulsar) | Python | Go(StreamNative) |
---|---|---|---|---|
InputTopics | ✅ | ✅ | ✅ | ✅ |
OutputTopic | ✅ | ✅ | ✅ | ✅ |
CurrentRecord | ✅ | ✅ | ✅ | ✅ |
OutputSchemaType | ✅ | ❌ | ✅ | ✅ |
Tenant | ✅ | ✅ | ✅ | ✅ |
Namespace | ✅ | ✅ | ✅ | ✅ |
FunctionName | ✅ | ✅ | ✅ | ✅ |
FunctionId | ✅ | ✅ | ✅ | ✅ |
InstanceId | ✅ | ✅ | ✅ | ✅ |
NumInstances | ✅ | ❌ | ✅ | ✅ |
FunctionVersion | ✅ | ✅ | ✅ | ✅ |
PulsarAdminClient | ✅ | ❌ | ❌ | ❌ |
GetLogger | ✅ | ❌ | ✅ | ✅ |
RecordMetrics | ✅ | ✅ | ✅ | ❌ |
UserConfig | ✅ | ✅ | ✅ | ✅ |
Secrets | ✅ | ❌ | ✅ | ✅ |
State | ✅ | ❌ | ❌ | ✅ |
Publish | ✅ | ✅ | ✅ | ✅ |
ConsumerBuilder | ✅ | ❌ | ❌ | ❌ |
Seek / Pause / Resume | ✅ | ❌ | ❌ | ❌ |
PulsarClient | ✅ | ❌ | ❌ | ❌ |
Other
Other | Java | Go(Pulsar) | Python | Go(StreamNative) |
---|---|---|---|---|
Resources | ✅ | ✅ | ✅ | ✅ |
At-most-once | ✅ | ✅ | ✅ | ✅ |
At-least-once | ✅ | ✅ | ✅ | ✅ |
Effectively-once | ✅ | ❌ | ✅ | ❌ |
Package
For Golang, we need to compile the function file to an executable one:-
Prepare your function file:
-
Compile
Deploy
After creating a cluster, set up your environment and develop&package your function, you can use thesnctl
, pulsarctl
, pulsar-admin
command, the REST API, or terraform
to deploy a Pulsar function to your cluster.
You can create a Golang Pulsar function by using a local compiled Golang file or an uploaded Pulsar functions package(recommend).
(Optional) Upload your function file to Pulsar
It’s recommend to upload your function file to Pulsar before you create a function. Since you can add a version suffix to the package.Upload packagesYou should see the following output:
Create
We are using a different Golang runtime, need to specify the
--custom-runtime-options '{"genericKind": "executable"}'
to make it work.What’s next?
- Learn how to develop NodeJs functions.
- Learn how to develop WASM functions.
- Learn how to manage functions.
- Learn how to monitor functions.
- Reference common configurations.