Skip to main content

​
StreamNative Weekly Release Notes v4.0.5.3

​
Download

​
Distributions

​
Packages

​
Images

​
General Changes

​
Apache Pulsar

(#24443) [fix][txn] Fix negative unacknowledged messages in transactions by ensuring that the batch size is added into CommandAck (#24421) [fix][build] Add missing name to submodules (#24459) [improve][broker] change to warn log level for ack validation error (#24441) [fix][ml] Enhance OpFindNewest to support skip non-recoverable data (#24454) [improve][broker][branch-4.0] Update to Oxia 0.6.0 and use new group-id (#24448) [refactor][broker] Expose the managedLedger field for the sub class

​
KoP

Add lookup cache for transaction marker channel manager

​
StreamNative Pulsar Plugins

Fix the build issue Upgrade zk version to 3.9.3 to avoid CVE-2024-51504 Bump com.fasterxml.jackson.core:jackson-core from 2.13.0 to 2.15.0 in sn-pulsar-tool

​
pulsarctl

(#1801) Enhance GetStatsCmd to include additional options for backlog statistics

​
StreamNative Tiered storage

[Branch-4.0] Cherry pick #1051 #1061

​
StreamNative Ursa storage

aed7cdec Use maven 3.9.9 settings 24cd9b7e Change command 036a9127 Change command a1f57661 change interactiveMode mode e6117b73 change interactiveMode mode 09d4c7c7 change interactiveMode mode 0680adc3 Add local maven settings to skip twitter downloading b023c907 Add local maven settings to skip twitter downloading Fix build script Refactor pulsar external table Support load pulsar client token from file Use the existing resources to init lakehouse reader Renable the pulsar e2e tests Get ledger metadata from Oxia 31275451 fix cherry-pick issue Change default entrySerDeType to PULSAR_BATCHED_RAW_PARQUET support register managedledger meatadata in oxia Pulsar support reading from parquet store messageId into parquet file Store ledger metadata to oxia Use jar instead of nar for offloader Add serialization type in the metadata Revert to use normal file as the index file Support save pulsar entry without parsing batch Support skip system topic and black topics Fix the bookkeeperApi can not get the index by secondary key Using MapFile to speed up the seek performance Support deleting the compacted data Update the offload flag according to the ursa storage state Make pulsar compaction worker enable iceberg external table writer Reuse the pulsar storage configuration for the pulsar offloader Update metadata store ledgers info after checking offloaded flag. Fix the prepare task name compatibility issue [cleanup] remove unused code in the managedLedgerWithTs Support generate the Ursa offset when committing the task Update to Oxia 0.6.0 and use new group-id Make bookkeeperStorageApi implement the StorageApi Pulsar offloader leader support Support write without parse content with schema Support compress the index files Make the tasks in the CommitTaskProvider sorted by the stream id and start offset Pulsar offloader support offload to iceberg format Support read/write index file with hadoop lib to cloud storage Fix pulsar lakehouse reader memory leak issue Integration test for the pulsar protocol compaction process Refactored to allow support for low-latency storage class Adapt the comapction process with pulsar related code Fix the NPE when serializing the bytes schema version Use static instances for compression codecs Pin version for commons-lang3 Support bookkeeper entry reader to let compaction service read from bookkeeper Support read and write bookkeeper entries Introduce the api for bookkeeper generate offsets Support trimming topic with the mark-deleted-offset

​
Security Fixes