Skip to main content

​
StreamNative Weekly Release Notes v4.1.0.4

​
Download

​
Distributions

​
Packages

​
Images

​
General Changes

​
Apache Pulsar

(#24863) [fix]Fixed getChildren(’/’) on Oxia based provider (#24852) [fix][ml] Fix getNumberOfEntries may point to deleted ledger (#24855) [fix][ml] Fix ledger trimming race causing cursor to point to deleted ledgers (#24838) [fix][broker] Ensure LoadSheddingTask is scheduled after metadata service is available again (#24841) [improve][ci] Upgrade GitHub Actions workflows to use ubuntu-24.04 (#24840) [improve] Upgrade Alpine base image to 3.22 version (#24836) [fix][ml] PIP-430: Fix concurrency issue in MessageMetadata caching and improve caching (#24830) [fix][client] Fix getPendingQueueSize for PartitionedTopicProducerStatsRecorderImpl: avoid NPE and implement aggregation (#24832) [fix] Fix mixed lookup/partition metadata requests causing reliability issues and incorrect responses (#24821) [fix][test] Fixed nondeterministic JSON ordering in multiple tests (#24798) [improve][client] PIP-420: Update the schema ID format (#24825) [improve][broker] Cache last publish timestamp for idle topics to reduce storage reads (#24815) [feat][monitor] Add ML write latency histogram and entry size histogram as OTel metrics (#24829) [fix][broker] Allow intermittent error from topic policies service when loading topics (#24822) [fix][client] Make auto partitions update work for old brokers without PIP-344 (#24810) [feat][monitor] Add publish latency histogram as OTel metrics (#24823) [fix][test] Fix flaky SingleThreadNonConcurrentFixedRateSchedulerTest.testPeriodicTaskCancellation (#24824) [improve][ml] Upgrade Oxia client to 0.7.0 (#24790) [feat][client] Implement PIP-234 for sharing thread pools and DNS resolver/cache across multiple Pulsar Client instances (#24734) [fix][client] Fix PulsarAdmin description check and add test (#24729) [improve][client] Allow adding custom description to User-Agent header (#24728) [fix][client] Add description method to ClientBuilder (#24812) [fix][build] Remove invalid profile in settings.xml that caused gpg signing to fail (#24811) [fix][build] Fix maven deploy with maven-source-plugin 3.3.1 (#24801) [improve][broker]Improve NamespaceService log that is printed when cluster was removed (#24785) [fix][broker] Fix incorrect topic loading latency metric and timeout might not be respected (#24770) [fix][broker] Flaky-test: ExtensibleLoadManagerImplTest.testDisableBroker (#24784) [improve][client/broker] Add DnsResolverGroup to share DNS cache across multiple PulsarClient instances (#24780) [improve][broker] Replace isServiceUnitActiveAsync with checkTopicNsOwnership (#24813) [fix] Update gRPC to 1.75.0

​
AoP

74655b2 Use commons-lang3

​
MoP

a57d06db update pulsar and sn dependency version

​
KoP

Handle invalid topic format error for metadata request feat(kafka-admin): Support describing topic configs with kop.kafka. prefix Use the Pulsar format for the Kafka consumer offsets topic fix(kafka-impl): correct topic partition extraction from Pulsar topic names Reduce spamming logs from schema registry and topic lookup [Ursa] Fix the wrong position comparison in topic replay 492502d4a Bump version to 4.1.0.4 Add producer ID expiration mechanism for classic engine Fix released produce request buffer could be accessed Make avro-maven-plugin version consistent with avro Use new APIs from Oxia 0.7.0 Write transaction log and offset log with partition log [schema-registry] add option to disable compatibility mode configuration Add more info to logs when the connection is closed Fix direct memory oom with ack=0 [Ursa] Improve producer state snapshot taking to avoid metadata thread Refactor schema provider to use Confluent avro schema provider [Ursa] Remove oldest producers when the serialized producer state is too large Support Pulsar and Kafka client communicate with the Kafka schema registry Support parse googles built in proto files on protobuf schema Fix idempotent producer for classic engine Improve topic loading time by skipping Pulsar message deduplication recovery Add test cases for record schema validation with subject name strategy

​
StreamNative Pulsar Plugins

Upgrade commons-lang to commons-lang3 Upgrade Oxia to 0.7.0 Upgrade zk version to 3.9.4 to fix CVE

​
Cloud Pulsar Plugins

5fd37f2 Use commons-lang3

​
Function Mesh Worker Service

Use sn-operator to deploy pulsar cluster in CI fix agent cannot update some fields error Fix ci failure fix build and license header

​
StreamNative Tiered storage

Use lang3 package StringUtils to replace lang package StringUtils Upgrade aws sdk to 2.32.28 to keep sync with Pulsar

​
StreamNative Ursa storage

Respect markDeletedOffsets when reading entries from PersistStorageApi Fix pulsar expired ledger not deleted bug Carry the number of messages info for positions in Entry Fix delta not support timestamp_ntz Do iceberg schema compatibility check before updating schema Use strict match rule to fetch token from UnityCatalogSasTokenProviderTest Disable flaky AsyncCleanerTest#simpleTest Fix parquet reader handle union type bug fix: support nested field partitioning in Iceberg tables Remove METADATA_DELETE_AFTER_COMMIT_ENABLED property Skip remove preserved properties Fix catalog close bug Add more log for parse entry Move the schema evolution logic into the iceberg table Upgrade Oxia client to 0.7.0 Bump org.apache.commons:commons-lang3 from 3.17.0 to 3.18.0 Add more admin commands Abstract the schema evolution parts in the encoder Removed dependency on commons-lang 2.6

​
Security Fixes