> ## Documentation Index
> Fetch the complete documentation index at: https://docs.streamnative.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Develop Pulsar Functions in Golang(Private Preview)

This section introduces how to develop and pacakge Golang Pulsar functions to use on StreamNative cloud.

We provide a different Golang runtime other than the community one, and it's still in private preview stage, If you want to try it out or have any questions, please [submit a ticket](https://support.streamnative.io/hc/en-us/requests/new) to the support team.

## Develop

We provide a GO [SDK](https://github.com/streamnative/pulsar-function-go) for developing Golang Pulsar Functions.

The following examples use Pulsar Functions SDK for the Golang language.

```go theme={null}
package main

import (
	"context"
	"fmt"
	"strings"

	"github.com/streamnative/pulsar-function-go/pf"
	"github.com/sirupsen/logrus"
)

func HandleExclamation(ctx context.Context, in []byte) ([]byte, error) {
	// 1. unmarshal []byte to your struct, use any schema you want
	payload := string(in)

	// 2. do your logic
	if fc, ok := pf.FromContext(ctx); ok {
		for _, word := range strings.Split(payload, " ") {
			// 2.1 Incr and Get Counter from state store
			_ = fc.IncrCounter(word, 1)
			count, _ := fc.GetCounter(word)
			// 2.2 Sending logs to a Pulsar topic
			logrus.Infof("got word: %s for %d times", word, count)
		}
        // 2.3 Get user-defined configurations
        cfg := fc.GetUserConfValue("configKey")
		// 2.4 Get secret configurations
        sec, err := fc.GetSecret("secretKey")
        if err == nil {
            msg := fmt.Sprintf("config: %v, secret: %s", cfg, *sec)
			// 2.5 Publish to any topic
            _, _ = fc.Publish("persistent://public/default/test-exec-package-serde-extra", []byte(msg))
        }
	}
	data := payload + "!"

	// 3. marshal your struct to []byte
	return []byte(data), nil
}

func main() {
	pf.Start(HandleExclamation)
}
```

To get more examples, please refer to [examples](https://github.com/streamnative/pulsar-function-go/tree/master/examples)

### Feature Matrix

The StreamNative's Golang runtime doesn't support full features comparing to Java runtime, and it's still in developing, below is the matrix:

#### Input Arguments

| Input                       | Java | Go(Pulsar) | Python | Go(StreamNative) |
| :-------------------------- | :--- | :--------- | :----- | :--------------- |
| Custom SerDe                | ✅    | ❌          | ✅      | **?**            |
| Schema - Avro               | ✅    | ❌          | ✅      | **?**            |
| Schema - JSON               | ✅    | ❌          | ✅      | **?**            |
| Schema - Protobuf           | ✅    | ❌          | ❌      | **?**            |
| Schema - KeyValue           | ✅    | ❌          | ❌      | **?**            |
| Schema - AutoSchema         | ✅    | ❌          | ❌      | **?**            |
| Scehma - Protobuf Native    | ✅    | ❌          | ❌      | **?**            |
| e-2-e encryption            | ✅    | ❌          | ✅      | ✅                |
| maxMessageRetries           | ✅    | ❌          | ❌      | ✅                |
| dead-letter policy          | ✅    | ❌          | ❌      | ✅                |
| SubscriptionName            | ✅    | ✅          | ✅      | ✅                |
| SubscriptionType            | ✅    | ✅          | ✅      | ✅                |
| SubscriptionInitialPosition | ✅    | ❌          | ✅      | ✅                |
| AutoAck                     | ✅    | ✅          | ✅      | ✅                |

<Note title="Note">
  Users can implement the Schema themselves since we are passing and expecting \[]byte to/from the users' function, so leave **?** here.
</Note>

#### Output Arguments

| Output                   | Java | Go(Pulsar) | Python | Go(StreamNative) |
| :----------------------- | :--- | :--------- | :----- | :--------------- |
| Custom SerDe             | ✅    | ❌          | ✅      | **?**            |
| Schema - Avro            | ✅    | ❌          | ✅      | **?**            |
| Schema - JSON            | ✅    | ❌          | ✅      | **?**            |
| Schema - Protobuf        | ✅    | ❌          | ❌      | **?**            |
| Schema - KeyValue        | ✅    | ❌          | ❌      | **?**            |
| Schema - AutoSchema      | ✅    | ❌          | ❌      | **?**            |
| Schema - Protobuf Native | ✅    | ❌          | ❌      | **?**            |
| useThreadLocalProducers  | ✅    | ❌          | ❌      | ✅                |
| Key-based Batcher        | ✅    | ✅          | ✅      | ✅                |
| e-2-e encryption         | ✅    | ❌          | ✅      | ✅                |
| Compression              | ✅    | ✅          | ✅      | ✅                |

#### Context

| Context               | Java | Go(Pulsar) | Python | Go(StreamNative) |
| :-------------------- | :--- | :--------- | :----- | :--------------- |
| InputTopics           | ✅    | ✅          | ✅      | ✅                |
| OutputTopic           | ✅    | ✅          | ✅      | ✅                |
| CurrentRecord         | ✅    | ✅          | ✅      | ✅                |
| OutputSchemaType      | ✅    | ❌          | ✅      | ✅                |
| Tenant                | ✅    | ✅          | ✅      | ✅                |
| Namespace             | ✅    | ✅          | ✅      | ✅                |
| FunctionName          | ✅    | ✅          | ✅      | ✅                |
| FunctionId            | ✅    | ✅          | ✅      | ✅                |
| InstanceId            | ✅    | ✅          | ✅      | ✅                |
| NumInstances          | ✅    | ❌          | ✅      | ✅                |
| FunctionVersion       | ✅    | ✅          | ✅      | ✅                |
| PulsarAdminClient     | ✅    | ❌          | ❌      | ❌                |
| GetLogger             | ✅    | ❌          | ✅      | ✅                |
| RecordMetrics         | ✅    | ✅          | ✅      | ❌                |
| UserConfig            | ✅    | ✅          | ✅      | ✅                |
| Secrets               | ✅    | ❌          | ✅      | ✅                |
| State                 | ✅    | ❌          | ❌      | ✅                |
| Publish               | ✅    | ✅          | ✅      | ✅                |
| ConsumerBuilder       | ✅    | ❌          | ❌      | ❌                |
| Seek / Pause / Resume | ✅    | ❌          | ❌      | ❌                |
| PulsarClient          | ✅    | ❌          | ❌      | ❌                |

#### Other

| Other            | Java | Go(Pulsar) | Python | Go(StreamNative) |
| :--------------- | :--- | :--------- | :----- | :--------------- |
| Resources        | ✅    | ✅          | ✅      | ✅                |
| At-most-once     | ✅    | ✅          | ✅      | ✅                |
| At-least-once    | ✅    | ✅          | ✅      | ✅                |
| Effectively-once | ✅    | ❌          | ✅      | ❌                |

## Package

For Golang, we need to compile the function file to an executable one:

1. Prepare your function file:

   ```go theme={null}
   package main

   import (
       "context"
       "github.com/streamnative/pulsar-function-go/pf"
   )

   func HandleExclamation(ctx context.Context, in []byte) ([]byte, error) {
       return []byte(string(in) + "!"), nil
   }

   func main() {
       pf.Start(HandleExclamation)
   }
   ```

2. Compile

   ```bash theme={null}
   go mod init func
   go mod tidy
   GO_ENABLED=0 GOOS=linux GOARCH=amd64 GO111MODULE=on go build -o exclamation exclamation.go
   ```

## Deploy

After creating a cluster, set up your environment and develop\&package your function, you can use the `snctl`, `pulsarctl`, `pulsar-admin` command, the REST API, or `terraform` to deploy a Pulsar function to your cluster.

You can create a Golang Pulsar function by using a local compiled Golang file or an uploaded Pulsar functions package(recommend).

### (Optional) Upload your function file to Pulsar

It's recommended to upload your function file to Pulsar before you create a function. Since you can add a version suffix to the package.

<Tabs>
  <Tab title="snctl">
    Upload packages

    ```bash theme={null}
    snctl pulsar admin packages upload function://${tenant}/${namespace}/${package_name} \
    --path ${file_path} \
    --description "${description}" \
    --properties fileName=${file_name}
    ```

    You should see the following output:

    ```bash theme={null}
    The package 'function://${tenant}/${namespace}/${package_name}' uploaded from path '${file_path}' successfully
    ```
  </Tab>

  <Tab title="Pulsarctl">
    You need to set the context for Pulsarctl first:

    ```bash theme={null}
    # create a context
    pulsarctl context set ${context-name}  \
    --admin-service-url ${admin-service-url} \
    --issuer-endpoint ${issuerUrl} \
    --audience urn:sn:pulsar:${orgName}:${instanceName} \
    --key-file ${privateKey}

    # activate oauth2
    pulsarctl oauth2 activate
    ```

    <Note title="Note">
      Replace the placeholder variables with the actual values that you can get when [setting up client tools](/cloud/process/pulsar-functions/function-setup#set-up-client-tools).

      * `context-name`: any name you want
      * `admin-service-url`: the HTTP service URL of your Pulsar cluster.
      * `privateKey`: the path to the downloaded OAuth2 key file.
      * `issuerUrl`: the URL of the OAuth2 issuer.
      * `audience`: the [Uniform Resource Name (URN)](/cloud/references/glossary#uniform-resource-name-urn), which is a combination of the `urn:sn:pulsar`, your organization name, and your Pulsar instance name.
        * `${orgName}`: the name of your [organization](/cloud/references/glossary#organization).
        * `${instanceName}`: the name of your [instance](/cloud/references/glossary#instance).
    </Note>

    Upload packages

    ```bash theme={null}
    pulsarctl packages upload function://${tenant}/${namespace}/${package_name} \
    --path ${file_path} \
    --description "${description}" \
    --properties fileName=${file_name}
    ```

    You should see the following output:

    ```bash theme={null}
    The package 'function://${tenant}/${namespace}/${package_name}' uploaded from path '${file_path}' successfully
    ```
  </Tab>

  <Tab title="Pulsar-admin">
    ```bash theme={null}
    ./bin/pulsar-admin \
        --admin-url "${WEB_SERVICE_URL}" \
        --auth-plugin org.apache.pulsar.client.impl.auth.oauth2.AuthenticationOAuth2 \
        --auth-params '{"privateKey":"file://${privateKey}","issuerUrl":"${issuerUrl}","audience":"urn:sn:pulsar:${orgName}:${instanceName}}' \
        packages upload function://${tenant}/${namespace}/${package_name} \
        --path ${file_path} \
        --description "${description}" \
        --properties fileName=${file_name}
    ```

    <Note title="Note">
      Replace the placeholder variables with the actual values that you can get when [setting up client tools](/cloud/process/pulsar-functions/function-setup#set-up-client-tools).

      * `admin-url`: the HTTP service URL of your Pulsar cluster.
      * `privateKey`: the path to the downloaded OAuth2 key file.
      * `issuerUrl`: the URL of the OAuth2 issuer.
      * `audience`: the [Uniform Resource Name (URN)](/cloud/references/glossary#uniform-resource-name-urn), which is a combination of the `urn:sn:pulsar`, your organization name, and your Pulsar instance name.
        * `${orgName}`: the name of your [organization](/cloud/references/glossary#organization).
        * `${instanceName}`: the name of your [instance](/cloud/references/glossary#instance).
    </Note>

    You should see the following output:

    ```bash theme={null}
    The package 'function://${tenant}/${namespace}/${package_name}' uploaded from path '${file_path}' successfully
    ```
  </Tab>
</Tabs>

### Create

<Tabs>
  <Tab title="snctl">
    ```bash theme={null}
    snctl pulsar admin functions create \
    --tenant public \
    --namespace default \
    --name function1 \
    --inputs persistent://public/default/test-go-input \
    --output persistent://public/default/test-go-output \
    --classname exclamation \
    --go function://public/default/go-exclamation@v0.1 \
    --custom-runtime-options '{"genericKind": "executable"}'
    ```

    <Note title="Note">
      We are using a different Golang runtime, need to specify the `--custom-runtime-options '{"genericKind": "executable"}'` to make it work.
    </Note>

    You should see something like this:

    ```bash theme={null}
    Created function1 successfully
    ```
  </Tab>

  <Tab title="Pulsarctl">
    ```bash theme={null}
    pulsarctl functions create \
    --tenant public \
    --namespace default \
    --name function1 \
    --inputs persistent://public/default/test-go-input \
    --output persistent://public/default/test-go-output \
    --classname exclamation \
    --go function://public/default/go-exclamation@v0.1 \
    --custom-runtime-options '{"genericKind": "executable"}'
    ```

    <Note title="Note">
      We are using a different Golang runtime, need to specify the `--custom-runtime-options '{"genericKind": "executable"}'` to make it work.
    </Note>

    You should see something like this:

    ```bash theme={null}
    Created function1 successfully
    ```
  </Tab>

  <Tab title="Pulsar-admin">
    ```bash theme={null}
    ./bin/pulsar-admin \
    --admin-url "${WEB_SERVICE_URL}" \
    --auth-plugin org.apache.pulsar.client.impl.auth.oauth2.AuthenticationOAuth2 \
    --auth-params '{"privateKey":"file:///YOUR-KEY-FILE-PATH","issuerUrl":"https://auth.streamnative.cloud/","audience":"urn:sn:pulsar:${orgName}:${instanceName}}' \
    functions create \
    --tenant public \
    --namespace default \
    --name function1 \
    --inputs persistent://public/default/test-go-input \
    --output persistent://public/default/test-go-output \
    --classname exclamation \
    --go function://public/default/go-exclamation@v0.1 \
    --custom-runtime-options '{"genericKind": "executable"}'
    ```

    <Note title="Note">
      We are using a different Golang runtime, need to specify the `--custom-runtime-options '{"genericKind": "executable"}'` to make it work.
    </Note>

    You should see something like this:

    ```bash theme={null}
    Created successfully
    ```
  </Tab>

  <Tab title="Terraform">
    Create your terraform yaml file:

    ```yaml theme={null}
    terraform {
      required_providers {
        pulsar = {
          version = "0.2.0"
          source = "registry.terraform.io/streamnative/pulsar"
        }
      }
    }

    provider "pulsar" {
      web_service_url = "{$admin-url}"
      api_version     = "3"
      audience = "urn:sn:pulsar:${orgName}:${instanceName}}"
      issuer_url = "${issuerUrl}"
      key_file_path = "${privateKey}"
    }

    // Note: function resource requires v3 api.
    resource "pulsar_function" "function-1" {
      provider = pulsar

      name = "function1"
      tenant = "public"
      namespace = "default"
      parallelism = 1

      processing_guarantees = "ATLEAST_ONCE"

      go = "function://public/default/go-exclamation@v0.1"
      classname = "exclamation.ExclamationFunction"

      inputs = ["persistent://public/default/test-go-input"]

      output = "persistent://public/default/test-go-output"

      subscription_name = "test-sub"
      subscription_position = "Latest"
      cleanup_subscription = true
      skip_to_latest = true
      forward_source_message_property = true
      retain_key_ordering = true
      auto_ack = true
      max_message_retries = 100
      dead_letter_topic = "public/default/dlt"
      log_topic = "public/default/lt"
      timeout_ms = 6666

      secrets = jsonencode(
      {
        "SECRET1": {
           "path": "sectest",
           "key": "hello"
        }
      })
      custom_runtime_options = jsonencode(
      {
         "genericKind": "executable",
          "env": {
              "HELLO": "WORLD"
          }
      })
    }
    ```

    <Note title="Note">
      We are using a different Golang runtime, need to specify the `--custom-runtime-options '{"genericKind": "executable"}'` to make it work.
    </Note>

    Init the terraform provider in the same dir of your `.tf` file if you haven't done it:

    ```bash theme={null}
    terraform init
    ```

    You should see something like this:

    ```bash theme={null}
    Initializing the backend...

    Initializing provider plugins...
    - Finding streamnative/pulsar versions matching "0.2.0"...
    - Installing streamnative/pulsar v0.2.0...
    - Installed streamnative/pulsar v0.2.0 (self-signed, key ID 3105E1011F3C3671)

    Partner and community providers are signed by their developers.
    If you'd like to know more about provider signing, you can read about it here:
    https://www.terraform.io/docs/cli/plugins/signing.html

    Terraform has created a lock file .terraform.lock.hcl to record the provider
    selections it made above. Include this file in your version control repository
    so that Terraform can guarantee to make the same selections by default when
    you run "terraform init" in the future.

    Terraform has been successfully initialized!

    You may now begin working with Terraform. Try running "terraform plan" to see
    any changes that are required for your infrastructure. All Terraform commands
    should now work.

    If you ever set or change modules or backend configuration for Terraform,
    rerun this command to reinitialize your working directory. If you forget, other
    commands will detect it and remind you to do so if necessary.
    ```

    Create the function:

    ```bash theme={null}
    terraform apply
    ```

    You should see something like:

    ```bash theme={null}
    Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
      + create

    Terraform will perform the following actions:

      # pulsar_function.function-1 will be created
      + resource "pulsar_function" "function-1" {
          + auto_ack                        = true
          + classname                       = "exclamation.ExclamationFunction"
          + cleanup_subscription            = true
          + cpu                             = 0.5
          + custom_runtime_options          = jsonencode(
                {
                  + genericKind = "executable",
                  + env = {
                      + HELLO = "WORLD"
                    }
                }
            )
          + dead_letter_topic               = "public/default/dlt"
          + disk_mb                         = 128
          + forward_source_message_property = true
          + id                              = (known after apply)
          + inputs                          = [
              + "persistent://public/default/test-go-input",
            ]
          + go                              = "function://public/default/go-exclamation@v0.1"
          + log_topic                       = "public/default/lt"
          + max_message_retries             = 100
          + name                            = "function1"
          + namespace                       = "default"
          + output                          = "persistent://public/default/test-go-output"
          + parallelism                     = 1
          + processing_guarantees           = "ATLEAST_ONCE"
          + ram_mb                          = 128
          + retain_key_ordering             = true
          + secrets                         = jsonencode(
                {
                  + SECRET1 = {
                      + key  = "hello"
                      + path = "sectest"
                    }
                }
            )
          + skip_to_latest                  = true
          + subscription_name               = "test-sub"
          + subscription_position           = "Latest"
          + tenant                          = "public"
          + timeout_ms                      = 6666
        }

    Plan: 1 to add, 0 to change, 0 to destroy.

    Do you want to perform these actions?
      Terraform will perform the actions described above.
      Only 'yes' will be accepted to approve.

      Enter a value:
    ```

    After enter "yes", you should see the following:

    ```bash theme={null}
    pulsar_function.function-1: Creating...
    pulsar_function.function-1: Creation complete after 1s [id=public/default/function1]

    Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
    ```

    <Note title="Note">
      Replace the placeholder variables with the actual values that you can get when [setting up client tools](/cloud/process/pulsar-functions/function-setup#set-up-client-tools).

      * `admin-url`: the HTTP service URL of your Pulsar cluster.
      * `privateKey`: the path to the downloaded OAuth2 key file.
      * `issuerUrl`: the URL of the OAuth2 issuer.
      * `audience`: the [Uniform Resource Name (URN)](/cloud/references/glossary#uniform-resource-name-urn), which is a combination of the `urn:sn:pulsar`, your organization name, and your Pulsar instance name.
        * `${orgName}`: the name of your [organization](/cloud/references/glossary#organization).
        * `${instanceName}`: the name of your [instance](/cloud/references/glossary#instance).
    </Note>
  </Tab>

  <Tab title="REST API">
    If you would like to create a function configuration using the REST API you can
    do so using CURL.

    ```bash theme={null}
    curl -X POST ${WEB_SERVICE_URL}/admin/v3/functions/public/default/${FUNCTION_NAME} \
      -H 'Authorization: Bearer ${TOKEN}' \
      -H "Content-Type: multipart/form-data" \
      -F 'functionConfig={"name": "${FUNCTION_NAME}", "tenant": "public", "namespace": "default", "runtime": "GO", "go": "function://public/default/${PACKAGE_NAME}@${PACKAGE_VERSION}", "output": "public/default/output-test", "inputs": ["public/default/input"], "className": "exclamation", "customRuntimeOptions": "{\"genericKind\": \"executable\"}"};type=application/json' \
      -F 'url=function://public/default/${PACKAGE_NAME}@${PACKAGE_VERSION}'
    ```

    <Note title="Note">
      We are using a different Golang runtime, need to specify the `--custom-runtime-options '{"genericKind": "executable"}'` to make it work.
    </Note>

    <Note title="Note">
      The function is assumed to be already uploaded at this point. If you have not
      uploaded the function, change the `url` parameter to be your local filepath.
      This will look something like the following.

      ```bash theme={null}
      -F 'url=file://$YOUR_LCOAL_FUNCTION_FILE'
      ```
    </Note>

    You should see something like this:

    ```bash theme={null}
    Created successfully
    ```

    <Note title="Note">
      Replace the placeholder variables with the actual values that you can get when [setting up client tools](/cloud/process/pulsar-functions/function-setup#set-up-client-tools).

      * `WEB_SERVICE_URL`: the HTTPS service URL of your Pulsar cluster.
      * `TOKEN`: a valid token to interact with your Pulsar cluster.
      * `FUNCTION_NAME`: the name of your function.
      * `FUNCTION_VERSION`: the version of your function you want to deploy e.g. 1.12.
    </Note>
  </Tab>
</Tabs>

For details about Pulsar function configurations, see [Pulsar function configurations](/cloud/process/pulsar-functions/function-config).

## What’s next?

* Learn how to develop [NodeJs functions](/cloud/process/pulsar-functions/develop-functions/function-develop-nodejs).
* Learn how to develop [WASM functions](/cloud/process/pulsar-functions/develop-functions/function-develop-wasm).
* Learn how to [manage functions](/cloud/process/pulsar-functions/function-manage).
* Learn how to [configure stateful functions](/cloud/process/pulsar-functions/function-state).
* Learn how to [monitor functions](/cloud/process/pulsar-functions/function-monitoring).
* Reference [common configurations](/cloud/process/pulsar-functions/function-config).