​
StreamNative Weekly Release Notes v4.0.1.1

​
Download

​
Distributions

​
Packages

​
Images

​
General Changes

​
Apache Pulsar

(#23824) [fix][client] Prevent retry topic and dead letter topic producer leaks when sending of message fails

(#22792) [improve][broker] Optimize subscription seek (cursor reset) by timestamp

(#23823) [fix][test] Remove useless test code

(#23817) [fix][broker] Remove failed OpAddEntry from pendingAddEntries

(#23813) [improve] Upgrade to Netty 4.1.116.Final and io_uring to 0.0.26.Final

(#23810) [improve][ci] Move ZkSessionExpireTest to flaky group to unblock CI

(#23784) [fix][admin] Fix exception thrown in getMessageId method

(#23776) [improve][ml] Optimize BlobStoreManagedLedgerOffloader.getOffloadPolicies

(#23779) [improve][txn] Improve Reader in TransactionBuffer to reduce GC pressure

(#23804) [improve][client] PIP-393: Support configuring NegativeAckPrecisionBitCnt while building consumer.

(#23666) [fix][test]: Flaky-test: GetPartitionMetadataMultiBrokerTest.testCompatibilityDifferentBrokersForNonPersistentTopic

(#23780) [improve][broker] Improve SystemTopicBasedTopicPoliciesService reader to reduce GC pressure

(#23802) [fix] [broker] Fix items in dispatcher.recentlyJoinedConsumers are out-of-order, which may cause a delivery stuck

(#23600) [improve][client] PIP-393: Improve performance of Negative Acknowledgement

(#23795) [fix][broker] Msg delivery is stuck due to items in the collection recentlyJoinedConsumers are out-of-order

(#23791) [fix][client] Cannot access message data inside ProducerInterceptor#onSendAcknowledgement

(#23786) [fix][broker] topic policy deadlock block metadata thread.

(#23652) [improve][log] Print ZK path if write to ZK fails due to data being too large to persist

(#23718) [fix][client] Make DeadLetterPolicy & KeySharedPolicy serializable

(#23798) [fix][client] Fix compatibility between kerberos and tls

(#23797) [fix][broker] Continue using the next provider for authentication if one fails

(#23781) [fix][broker] Fix enableReplicatedSubscriptions

(#23757) [improve][client] Make replicateSubscriptionState nullable

(#23772) [fix][ml] Topic load timeout due to ml data ledger future never finishes

(#23767) [fix][broker] System topic should not be migrated during blue-green cluster migration

(#23766) [fix][admin] Fix exception loss in getMessageId method

(#23762) [fix] Fix issues with Pulsar Alpine docker image stability: remove glibc-compat

(#23753) [fix][client] Fix enableRetry for consumers using legacy topic naming where cluster name is included

(#23693) [fix][client] Fix reader message filtering issue during blue-green cluster switch

(#23764) [fix][broker] Fix bug causing loss of migrated information when setting other localPolicies in namespace

(#23747) [fix][test] Fix flaky KeySharedSubscriptionTest.testNoKeySendAndReceiveWithHashRangeAutoSplitStickyKeyConsumerSelector

(#23761) [Fix][Client] Fix pending message not complete when closeAsync

(#23756) [improve][monitor] Upgrade OTel to 1.45.0

(#23738) [fix] [client] Fix memory leak when publishing encountered a corner case error

(#23734) [improve][fn] Improve closing of producers in Pulsar Functions ProducerCache invalidation

(#23708) [improve][fn] Improve implementation for maxPendingAsyncRequests async concurrency limit when return type is CompletableFuture

(#23752) [improve] Upgrade lombok to 1.18.36

(#23691) [fix][common] TopicName: Throw IllegalArgumentException if localName is whitespace only

(#23730) [fix][admin] Verify is policies read only before revoke permissions on topic

​
AoP

c11cb32 Fix build script

(#1433) Use SN bom

(#1471) remove useless check in tests

​
MoP

(#1590) Remove yahoo dependency

​
KoP

Revert “Revert using sn-bom (#957)“

​
Cloud Storage Connector

(#1172) Add configuration to exclude topic on file path

(#1142) Refactor thread mode and batch memory control

​
AMQP1_0 Connector

(#1092) Use new clean disk job

​
StreamNative Pulsar Plugins

fix azure package failed when list non-exists directory

Support Topic Level Tracing and LogTraceRecorder

Azure Blob Storage backed Package Management Service

Exclude commons-io to fix cve

Fix license commons plugin jar package

Use WebIdentity way when AWS_WEB_IDENTITY_TOKEN_FILE exists

fix test in BrokerUnloadJobResourcesTest

​
Cloud Pulsar Plugins

Use new clean disk job

​
Function Mesh Worker Service

Use large runner to avoid disk full issue

Fix free disk job of ubuntu-latest runner

bump kafka dep to 3.9.0

c4fd73a7 Use separate image for kafka connect in CI

Replace the deprecated getZooKeeperSessionTimeoutMillis

​
Google BigQuery Sink Connector

Stop executor when connector closed

​
StreamNative Unified RBAC

fix: fix the license of generator

feat: support new permission pulsar.packages.admin

feat: format generator license

feat: improve the document

Add User, ClusterRole, RoleBinding tests

fix(sdk-java): fix empty cel passed authorization

feat(sdk-go-cloud): upgrade the sdk-go to v0.2.0

fix(*): fix license

fix(*): fix license issue

feat(*): upgrade the project version to 1.4.0

feat(sdk-js): upgrade version to 0.0.8

all: basic SRN supprot.

Add application tests

Optimize integration tests

feat(sdk-js): resource admin support

Add more integration tests

​
StreamNative Ursa storage

Fix flaky test TestCompactionServiceBaseFileStorage#simple

Fix ManagedCursorTest#testReadEntriesWithSkipDeletedEntries flaky test.

Fix StorageWalManagedCursor#internalReadEntries only read one entry at each round.

Fix s3 compact flaky test.

[TEST] Run integration test with image ksn 3.9.0

Fix parquet prefetch bug

Add iceberg azure dependency

Use separate thread pool for parquet reading

Test master CI

Fix default compaction thread bug

Support load credential from file

add cloud region

bdb91385 [Bug] fix duplicated entry id put in PersistCache.index

Adjust the compaction default configuration according to the performance test

Fix flaky test in TestGarbageCollection

Add the ProfileCredentialsProvider into the auth chain

Disable nonblocking dns and get rid of request limitation of S3 prefix

improvement in dispatch logic

Fix the default write buffer segment configuration

Unity catalog support config User-agent.

Adding stress-ng to CI

Fix oxia read failed with small range

Improve CI by running S3 and GCS integration tests In parallel

Optimize PersistCache serialization to only persist used segments

Support disable read cache expire by time

Compaction service support GCS

Delta table support partition column using topic partition index

Compact service support azure blob protocol

Parse the storage account name and blob container name from bucket for azure

Check the table whether register to the unity catalog when commit action. If not, register the table to unity catalog.

Compaction service support Azure storage.

Modify delta table mapping name in unity catalog

Rename azure to azureblob to make it more precise

fix azure make data in heap

Quarantine the compact task if it read WAL data fialed.

Add the storage metrics for all the file storage type

Allow to disable the lakehouse reader in the managedledger

Enable ChecksumCRC32C for getting object from S3

​
Security Fixes

​
Apache Pulsar

(#23743) [fix][sec] Upgrade golang.org/x/crypto from 0.21.0 to 0.31.0 in pulsar-function-go