streamnative.io
Help Center
Login
  1. StreamNative Cloud
  2. Get Started

Quick Start for StreamNative Cloud

StreamNative Cloud is a resilient, scalable, data streaming service powered by the URSA engine, delivered as a fully managed Pulsar and Kafka service.

StreamNative Cloud provides multiple interfaces for management and interaction:

  1. StreamNative Cloud Console: A user-friendly web-based interface for managing cluster resources, configuring settings, and handling billing.

  2. Command-Line Interface (CLI):

    • StreamNative CLI (snctl): For creating and managing StreamNative Cloud resources, such as instances, clusters, service and user accounts, and more.
    • Pulsar CLI (pulsarctl): For managing cluster-specific resources, such as tenants, namespaces, and topics.

  3. REST APIs: For programmatic access and integration with other systems.

These tools provide flexibility in how you interact with and manage your StreamNative Cloud environment, catering to different user preferences and use cases.

Get started for Free

Sign up for StreamNative Cloud and get $200 of free credits. No credit card required.

This quick start guides you through getting started with StreamNative Cloud using a Serverless cluster. It demonstrates how to use StreamNative Cloud to create topics, produce data to the cluster, and consume data from it.

Note

This QuickStart assumes you are familiar with the basics concepts of Apache Pulsar.

Prerequisites

  • Access to StreamNative Cloud.

  • Internet connectivity.

  • Ensure you have installed Python 3.0 or higher versions and the Pulsar Python client.

    python -m pip install pulsar-client

Step 1: Sign up

Note

If you have an email account configured for using Single Sign-On (SSO) with StreamNative Cloud, use that email address and password when signing up.

To sign up, navigate to the StreamNative Cloud Console login page. Follow the prompts to create an account.

After you click Finish, you might have to wait briefly for your first organization to be created.

After your new organization is created, continue on to creating your first instance and cluster.

Step 2: Create a StreamNative instance and cluster

You first need to create an instance and a cluster.

  1. In the upper-right corner of the StreamNative Cloud Console, click your Profile and select Organizations to list your created organizations.

  2. Click the name of your first organization. In the figure below, the organization name is Demos.

    screenshot of organization section

  3. On the Select an Instance card of the Organization Dashboard page, click the + button.

  4. On the Choose the deployment type for your instance, click Deploy Serverless to start the instance creation process.

  5. On the Instance Configuration page, enter a name for your instance, and select a Cloud Provider. The instance name starts with a lowercase letter, contains any combination of lowercase letters (a-z), numbers (0-9), and hyphens (-), and must be 4-10 characters.

  6. Click Cluster Location to start the cluster creation process.

  7. On the Cluster Location page, enter a name for your cluster, select the cluster location, and then click Finish. The cluster name starts with a lowercase letter, contains any combination of lowercase letters (a-z), numbers (0-9), and hyphens (-), and must be 4-10 characters.

The cluster page appears, displaying the cluster creation process. Depending on the chosen cloud provider and other settings, it may take a few seconds to several minutes to provision the cluster. Once the cluster has been provisioned, the Cluster Dashboard page will be displayed.

Now you can get started configuring apps and data on your new cluster.

Step 3: Create a service account

To interact with your cluster by producing and consuming messages, you need to set up authentication and authorization. This is done by creating a Service Account, which serves as an identity for authenticating and authorizing access to the cluster. The service account will provide the necessary credentials for your applications to securely connect and perform operations on the Pulsar cluster.

Note

Currently, you can't edit a service account. If you need a service account to have Super Admin access, make sure to enable it when creating the service account. By default, service accounts do not have Super Admin enabled.

  1. In the upper-right corner of the StreamNative Cloud Console, click your Profile and select Accounts & Accesses.

  2. On the left navigation pane, click Service Accounts.

  3. On the Service Account page, click + New Service Account.

  4. On the Create Service Account dialog, enter a name for the service account, and then click Confirm.

  5. On the Service Account page, in the row of the service account you just created, click the ... icon, and select Create API Key in the dropdown menu.

  6. On the New API Key dialog:

    • Enter a name for the API key
    • Set the expiration date
    • Select the instance you created in Step 2
    • Write a description for the API key
    • Click Confirm

    Note

    An API key and associated secret apply to the active StreamNative instance. If you add a new instance, you must create a new API key for producers and consumers on the new Pulsar instance. For more information, see Use API Keys to Authenticate to StreamNative Cloud.

  7. After the API key is created, you can see the API key shown in the New API Key dialog. Click the Copy and close button to copy the API key to your clipboard. Please note that you cannot retrieve the API key later after closing the dialog.

Step 4: Create Tenant and Namespace, and Authorize the Service Account

After creating the service account and obtaining the API key, the next crucial step is to authorize the service account. This process grants the necessary permissions for the service account to interact with your StreamNative Cloud cluster.

Authorization involves setting up Access Control Lists (ACLs) that define what actions the service account can perform. Typically, you'll want to grant permissions for producing and consuming messages on specific topics or namespaces.

For more information about authorization, see Authorization and ACLs.

  1. Go back to the Cluster Dashboard page.

  2. On the left navigation pane, click Instances.

  3. On the Instances page, click the name of the instance you created in Step 2.

  4. On the Cluster Dashboard page, click Tenants on the left navigation pane.

  5. On the Tenants page, click + New Tenant.

  6. On the New Tenant dialog:

    • Enter a name for the tenant
    • Select your user account as the Admin role
    • Select the cluster created in Step 2 as Allowed clusters
    • Click Confirm

  7. On the Tenants page, click the name of the tenant you just created. You will be directed to the Tenant Dashboard page.

  8. On the Tenant Dashboard page, click Namespaces on the left navigation pane.

  9. On the Namespaces page, click New Namespace.

  10. On the New Namespace dialog:

    • Enter a name for the namespace
    • Select the cluster created in Step 2 as Allowed clusters and Replication clusters
    • Click Confirm

  11. On the Namespaces page, click the name of the namespace you just created. You will be directed to the Namespace Dashboard page.

  12. On the Namespace Dashboard page, click Configuration on the left navigation pane.

  13. On the Namespace configuration page, click ADD ROLE. Select the name of the service account you just created, and choose the consume and produce permissions. This grants your service account the produce and consume permissions for this namespace.

Now you have created a tenant, namespace, and granted the service account the produce and consume permissions for this namespace. You can now continue on to building a Python app, connecting to the cluster, and producing and consuming messages.

Step 5: Produce and consume messages

Note

This QuickStart provides you with an example Python app to get you up and running with consuming and producing messages. This is a simple example and is not intended for production environments.

Create a producer/consumer

  1. Return to the StreamNative Cloud Console and go to the "Cluster Dashboard" page.

  2. On the left navigation pane, click Pulsar Clients.

  3. On the Pulsar client setup page, click the Code libraries tab, follow the setup wizard to get the sample codes for your producer and consumer.

    a. Select Python as the client library and click Next.

    b. Select the service account you created and click Next.

    c. Select API Key as the authentication type and click Next.

    d. Install pulsar-client python client library.

    pip3 install pulsar-client

    e. Select the target tenant, namespace, topic, and subscription.

    f. You are now ready to copy the auto-generated sample codes.

  4. Return to your text editor and create two new files: producer.py and consumer.py. Copy and paste the sample code for the producer into producer.py and the sample code for the consumer into consumer.py. In both files, replace <JWT Token> with the API key you copied from the Service Account page.

Run the clients to produce and consume your first message

  1. Open a terminal window, navigate to the folder containing the consumer.py file, and run the following command:

    python3 consumer.py

  2. Open a second terminal window, navigate to the folder containing the producer.py file, and run the following command:

    python3 producer.py

  3. Return to the first terminal window. You should see the following:

    Received message 'Hello-0' id='<message-id>'
Received message 'Hello-1' id='<message-id>'
Received message 'Hello-2' id='<message-id>'
Received message 'Hello-3' id='<message-id>'
Received message 'Hello-4' id='<message-id>'
Received message 'Hello-5' id='<message-id>'
Received message 'Hello-6' id='<message-id>'
Received message 'Hello-7' id='<message-id>'
Received message 'Hello-8' id='<message-id>'
Received message 'Hello-9' id='<message-id>'

You have now produced and consumed your first 10 messages.

Next steps

  • After you have successfully provisioned a Serverless cluster and connected to the cluster, you can learn more about working with StreamNative Cloud by reading through Cloud Console basics.

  • If you want to learn more about Pulsar, Kafka, and StreamNative Cloud, take our developer courses at the StreamNative Developer Portal.

  • If you want to create multiple Pulsar clusters replicating messages with geo-replication mechanism, you can learn more by reading through the Geo Replication on StreamNative Cloud.

Previous
Engines
Next
Kafka APIs