Classic Engine
Classic Engine is currently the default engine for StreamNative Cloud.
Ursa Engine
Ursa Engine is currently available as a General Availability feature on AWS and Public Preview on Azure and GCP .
- Fully Kafka API-compatible: The Ursa Engine is fully compatible with the Kafka API, making it easier to migrate existing Kafka-based applications to Pulsar.
- Lakehouse storage for long-term durability and open standards-based storage.
- Oxia as a scalable and durable metadata storage.
- Support for both Latency-Optimized and Cost-Optimized storage options.
- Latency-Optimized storage: This storage option is based on Apache BookKeeper and provides low-latency data persistence. It is suitable for latency-sensitive workloads that require high-throughput and low-latency messaging and streaming.
- Cost-Optimized storage: This storage option is based on Object Storage (such as S3, GCS, and Azure Blob Storage) and provides a more cost-effective storage solution for workloads that can tolerate sub-second latencies in exchange for lower total cost of ownership.
What is included in the Ursa Engine Public Preview?
With the Public Preview, users can access core Ursa Engine Features such as Oxia-based metadata management and S3-based WAL, targeting cost-sensitive, latency-relaxed workloads. However, there are certain limitations of this Public Preview as shown below:- Only Kafka protocol is enabled during Public Preview. Pulsar & MQTT protocols are disabled.
- Transactions and topic compaction are not yet supported.
- Only S3-based Cost-Optimized storage is available, making it ideal for latency-relaxed use cases.
- For ultra-low-latency applications, we recommend continuing with the Classic Engine until the Ursa Engine reaches full GA.
Ursa Stream Storage
At the heart of the Ursa Engine is the concept of Ursa Stream Storage. It is a headless, multi-modal data storage layer built on lakehouse formats. At the heart of Ursa Stream Storage is a WAL (Write-Ahead Log) implementation based on S3. This design writes records directly to object storage services like S3, bypassing BookKeeper and eliminating the need for replication between brokers. As a result, Ursa Engine-powered clusters replace expensive inter-AZ replication with cost-efficient, direct-to-object-storage writes. This trade-off introduces a slight increase in latency (from 200ms to 500ms) but results in significantly lower network costs—on average 10x cheaper.
Comparison between Classic Engine and Ursa Engine
Below is a feature comparison between the Classic and Ursa Engines:Feature | Classic Engine | Ursa Engine (Public Preview) |
---|---|---|
Kafka Protocol | Yes | Yes, Transaction & Topic Compaction are not supported |
Pulsar Protocol | Yes | Disabled during Public Preview |
MQTT Protocol | Yes | Disabled during Public Preview |
Metadata Storage | ZooKeeper | Oxia |
Lakehouse Storage | Add-on | Built-in |
Latency-Optimized (BookKeeper-based) Storage | Yes | Disabled during Public Preview |
Cost-Optimized (S3-based) Storage | No | Yes |