- Manage Data Streams
Manage Data Streams in StreamNative Cloud
StreamNative Cloud leverages Apache Pulsar, a multi-tenant messaging and data streaming platform. Within Apache Pulsar, data streams are organized into Topics, which are grouped into Tenants and Namespaces. Tenants can span multiple clusters within a StreamNative Instance, and each tenant serves as an administrative unit where storage quotas, message TTL (Time to Live), isolation policies and etc. are managed.
The structure of topic URLs in Pulsar, which highlights its multi-tenant architecture, is as follows:
[persistent|non-persistent]://tenant/namespace/topic
In this structure, the tenant is the primary categorization unit for topics, more fundamental than the namespace or topic name itself.
You can create and manage tenants, namespaces, topics, and their configurations and policies through the StreamNative Cloud console, the Pulsar CLI (pulsarctl), or Pulsar Admin APIs.
Tenants
A Pulsar tenant is an administrative entity used for capacity allocation and implementing authentication or authorization schemes. Each tenant in a Pulsar instance can have:
- An assigned authorization scheme
- A defined set of clusters where the tenant's configurations are applicable
Namespace
A namespace in Pulsar is a logical grouping of topics. It serves as an administrative segment within a tenant, where configuration policies set at the namespace level apply to all topics within that namespace. A tenant can establish multiple namespaces to accommodate different applications, for example:
persistent://tenant/app1/topic-1
persistent://tenant/app2/topic-2
persistent://tenant/app3/topic-3
Topic
A topic in Pulsar is a storage unit that organizes messages into a stream. Similar to other publish-subscribe systems, topics in Pulsar act as named channels that facilitate message transmission from producers to consumers. The naming convention for topics follows a specific URL pattern:
{persistent|non-persistent}://tenant/namespace/topic
The components of a topic name are defined as:
| Topic name component | Description |
|---|---|
persistent / non-persistent | Indicates the type of topic. Pulsar supports both persistent and non-persistent topics. Persistent topics save all messages on disk, ensuring durability (messages are stored on multiple disks unless the broker is standalone). Non-persistent topics do not save data on disks. |
tenant | Represents the tenant within the instance, highlighting Pulsar's support for multi-tenancy. |
namespace | Acts as a grouping mechanism for related topics, with most configuration done at this level. Each tenant may have one or more namespaces. |
topic | The specific name of the topic within a namespace. Topic names in Pulsar do not carry intrinsic meanings beyond their hierarchical organization. |
Relate topics
This section could include links or references to further reading on related topics or advanced configurations in Pulsar.