Skip to main content

StreamNative Weekly Release Notes v5.0.0.3

Download

Distributions

Packages

Images

General Changes

Apache Pulsar

(#27321) [fix][meta] Fix leader elections settling on a lock deleted while they read or write it (#27112) [fix][client] Wait for the first key load before completing the producer creation (#27341) [improve][test] Speed up five Broker Group 1 test classes: transactions, lost ledgers and BrokerService (#27329) [fix][test] Fix flaky SubscriptionSeekTest.testSeekWillNotEncounteredFencedError (#27338) [improve][test] Run the Messaging integration group’s suites in one Gradle run (#27337) [improve][test] Let integration test classes that ask for the same cluster share one (#27334) [improve][test] Speed up four Broker Group 1 test classes (#27202) [fix][broker] Skip bundle queries for cancelled ALL topic listings (#27333) [fix][client] Keep the Backoff jitter from taking a retry past the mandatory stop (#27200) [fix][client] Send pending acks before listener and resume permits on reconnect (#27331) [fix][test] Fix ServerCnxTest.testSendWithoutSchemaVersionFromAutoProduceBytesProducer failing since #27314 (#27147) [fix][ml] Retry a failed read on the bookies a cached read handle’s ledger was re-replicated to (#27158) [fix][broker] Retry applying received subscription updates (#27326) [fix][test] Avoid premature batch flushes in ProduceWithMessageIdTest (#27278) [fix][client] Don’t leak flow permits when a multi-topics consumer with a listener drops queued messages (#27272) [fix][broker] Make deleting a scalable topic subscription work on ZooKeeper and Oxia (#27322) [improve][test] Speed up ten Broker Group 1 test classes (#27320) [fix][broker] Don’t make clients retry a partitioned topic auto-creation that exceeds maxNumPartitionsPerPartitionedTopic (#27319) [improve][test] Speed up PrometheusMetricsTest.testPublishRateLimitedTimes (#27318) [improve][test] Speed up BrokerBookieIsolationTest.testBookieIsolationWithSecondaryGroup (#27317) [fix][broker] Answer lookups waiting for a bundle owner when the table view refills after a session is reestablished (#27316) [improve][test] Speed up BacklogQuotaManagerTest (#27315) [fix][test] Fix flaky TransactionClientCloseTest.testCloseDoesNotWaitForTransactionCallbacks (#27314) [fix][broker] Fix replication stuck on a schema-less message to a remote topic with an enforced schema (#27146) [fix][client] Preserve batch-index ACKs during concurrent flush (#27292) [fix][ml] Restore best-effort per-cursor offline backlog estimation (#27301) [fix][client] Don’t return pre-redeliver messages from a multi-topics consumer’s receive() (#27307) [improve][broker] Dispatch a loaded bucket snapshot segment without waiting for the next tick (#27312) [fix][broker] Fail a schema-less AUTO_PRODUCE_BYTES send at once instead of retrying it until the send timeout (#27313) [improve][test] Don’t wait for the event loop quiet period when closing InjectedClientCnxClientBuilder clients (#27299) [improve][ci] Balance the unit test groups’ forks by recorded test class durations (#27304) [improve][test] Speed up the BookKeeper replication tests in pulsar-metadata (#27309) [improve][test] Speed up BucketDelayedDeliveryTrackerTest (#27306) [improve][test] Speed up PersistentQueueE2ETest (#27308) [improve][test] Speed up ServiceUnitStateChannelTest (#27305) [improve][ci] Shorten CI - Unit - Other: start the jcloud tests first, skip JVM-wide threads in the leak detector (#27303) [improve][test] Speed up six Client Api test classes that wait out fixed timers (#27291) [fix][broker] Close and evict topics that failed to unload when the bundle is released (#27302) [fix][test] Fix flaky PulsarFunctionsJavaTest window tests: wait for the function’s consumer before producing (#27289) [fix][proxy] Keep proxy stats updates running after a failed run (#27290) [fix][ml] Complete automatic offload when its background task fails (#27298) [improve][test] Speed up the ack-timeout, negative-ack and DLQ tests (#27296) [fix][test] Fix KeySharedSubscriptionTest hang caused by a leaked default message TTL (#27297) [improve][test] Speed up KeySharedSubscriptionTest (#27099) [fix][fn] Log from close() when the function instance process is stopped (#27105) [fix][broker] Refresh stale partition metadata before rejecting a partition (#27295) [improve][test] Speed up TransactionEndToEndTest (#27179) [fix][ci] Upgrade gradle/actions to 6.4.1 (GHSA-chpw-2mqm-95cc) and develocity-actions to 2.5 (#27260) [improve][broker] Close the broker’s connections before shutting down its event loops (#27169) [fix][client] Fail TC start when txn is disabled (#27217) [fix][client] Return only readable bytes from heap messages (#27212) [fix][ml] Close abandoned managed ledger initialization resources (#27210) [fix][client] Preserve batch boundaries and recover after failures (#27201) [fix][meta] Release closed caches and preserve migration listeners (#27279) [fix][broker] Make package metadata updates visible at once on every broker (#27275) [fix][broker] Return 409 instead of 412 when create-subscription requests overlap (#27273) [fix][broker] Close the producer of a geo-replicator stopped while it starts (#27261) [improve][test] Stop the WebSocket proxy before the broker in the websocket proxy tests (#27259) [fix][test] Keep AdminMultiHostTest’s and OpenTelemetrySanityTest’s container names within a DNS label (#27253) [fix][test] Run the non-persistent messaging integration tests on non-persistent topics (#27247) [fix][test] Cut ExtensibleLoadManagerTest integration test setup time (#27245) [fix][meta] Apply the migration phase to DualMetadataCache writes (#27237) [improve][client] Don’t wait for the event loop quiet period when closing a client (#27164) [fix][client] Fix key-based batch sort overflow (#27161) [fix][client] Keep cumulative retry properties (#27160) [fix][client] Keep the transaction of a cumulative ack on a multi-topics consumer (#27159) [fix][broker] Retry failed replicated subscription updates while idle (#27262) [improve][fn] Make a function worker become the leader without waiting out its tailers’ polls, and speed up the function worker tests (#27276) [improve][test] Cut fixed waits and setup CLI runs from the remaining Cli integration tests (#27274) [improve][test] Speed up the Go and Python function integration tests (#27269) [fix][test] Check the broker that serves a topic in OpenTelemetrySanityTest’s Prometheus export test (#27267) [fix][fn] Don’t fail a function worker’s start when another worker creates the coordination subscription first (#27266) [fix][proxy] Close the proxy extensions when the proxy shuts down (#27265) [improve][test] Use ZooKeeper’s Netty client socket in the pulsar-broker tests (#27264) [improve][proxy] Shut down the proxy’s event loops without waiting out Netty’s 2 s quiet period (#27258) [fix][test] Fix the HealthCheckTest and CLITest methods that fail their first attempt in every CI run (#27257) [fix][test] Fix flaky ReplicatorTest.testReplicatorProducerClosing (#27255) [improve][test] Speed up OneWayReplicatorDeduplicationTest by waiting on conditions instead of receive timeouts (#27271) [improve][test] Start and stop the integration tests’ Pulsar cluster in parallel groups (#27270) [fix][test] Fix flaky PulsarShellTest.testInterruptRunningCommand (#27256) [improve][test] Speed up ReplicatorTest by waiting on conditions instead of fixed waits (#27268) [fix][test] Wait for the function instance to subscribe before producing in the function state tests (#27280) [fix][cli] End pulsar-perf produce as soon as its test threads are done (#27248) [fix][client] Avoid hand-offs to terminated event loops while a client shuts down (#27240) [improve][test] Cut AdminApiTest and IncrementPartitionsTest setup and cleanup time (#27246) [fix][test] Fix flaky ManagedCursorTest.testReplayRangeReadsFitTheInflightReadsLimitWithLedgersOfDifferentEntrySizes (#27249) [fix][test] Fix flaky BacklogQuotaManagerTest.testConsumerBacklogEvictionSizeQuota (#27252) [improve][test] Speed up MessagingSmokeTest and the topic messaging integration tests (#26798) [fix][broker] Prevent incorrect isolation fallback during cluster updates (#27254) [improve][test] Speed up ConsumerBatchReceiveTest (#27250) [improve][test] Speed up the function state integration tests (#27244) [fix][build] Upgrade Lombok to 1.18.48 to fix @SneakyThrows on JDK 26 (#26380) [fix][txn] Stop pending ack replay from hot-looping on read failures (#27205) [fix][ml] Preserve cursor position updates during activation (#27101) [fix][cli] Don’t throw from the pulsar-shell signal handler on Ctrl-C (#27209) [fix][client] Prevent transaction callbacks from blocking client shutdown (#27213) [fix][broker] Prevent fixed delay detection from blocking messages (#27167) [fix][client] Check closed state before lazy start (#27150) [improve][test] Reduce BrokerServiceTest shutdown test wall time from 41s to 12s (#27171) [fix][admin] Keep sharedResources in clone() (#27151) [improve][test] Reduce BacklogQuotaManagerTest wall time from 387s to 292s (#27069) [fix][client] Prevent message loss from premature ACKs during batch DLQ forwarding (#27106) [fix][client] Reschedule the negative ack timer when a nack is due earlier (#27238) [improve][test] Speed up OneWayReplicatorTest by waiting on conditions instead of fixed sleeps (#27170) [fix][client] Keep ByteBuffer TableView values (#27168) [fix][client] Honor reader autoUpdatePartitions (#27154) [improve][test] Reduce ConsumerBatchReceiveTest async wall time from 186s to 43s (#27203) [fix][client] Resume sending after messages awaiting schema time out (#27153) [improve][test] Reduce AuthenticatedTransactionProducerConsumerTest wall time from 263s to 207s (#27241) [improve][test] Speed up SimpleProducerConsumerTest (#27243) [improve][broker] Close the broker’s own admin client before stopping its web service (#27242) [improve][test] Speed up testCloseTopicAfterStartReplicationFailed from 74 s to 2 s (#27163) [fix][client] Close interceptors once after growth (#27162) [fix][client] Await each batch reconsumeLater (#27236) [fix][meta] Select the DualMetadataCache store by migration phase (#27234) [improve][test] Cut AdminApi2Test setup and cleanup time (#27235) [improve][test] Share the clusters of ClusterMigrationTest across its test methods (#27219) [improve][test] Reduce TransactionConsumeTest wall time from 230s to 122s (#27233) [fix][test] Quiet the per-iteration logs of BookieRackAffinityMappingTest’s deadlock test (#27232) [fix][test] Shut down bookies and clients leaked by the pulsar-metadata replication tests (#27230) [improve][test] Log the jcloud offloader tests at INFO instead of DEBUG (#27231) [fix][test] Keep bookkeeper-server’s bundled log4j2.xml from replacing the shared test logging config (#27094) [improve][test] Reduce AuditorBookieTest wall time from 180s to 69s (#27198) [fix][txn] Balance pending writes when publish callbacks throw (#27197) [fix][broker] Prevent duplicate delivery from queued dispatches after reconnect (#27207) [fix][txn] Release late entries after snapshot recovery stops waiting (#27206) [fix][broker] Preserve commands across broker interceptors (#27214) [fix][broker] Prevent mixed-durability subscription creation races (#27215) [fix][ml] Fix managed ledger factory shutdown races and timeouts (#27226) [improve][meta] Upgrade Oxia Java client to 0.10.4 (#27111) [fix][broker] Support schema-less auto-produce registration (#27196) [improve][ci] Reduce disk usage and extend Broker Group 1 timeout in Pulsar CI (#27143) [improve][broker] Avoid regex splitting in topic-list filtering (#27194) [fix][test] Fix flaky ManagedLedgerTest.testManagedLedgerWithCreateLedgerTimeOut (#27152) [fix][test] Fix flaky DeadLetterTopicTest.testPossibleToDeadLetterEntryIsRemovedWhenRedeliveringABatchMessageId (#27182) [improve][build] Upgrade gRPC to 1.85.1 (#27180) [improve][misc] Upgrade Jetty to 12.1.14 and Conscrypt to 2.7.0 (#27177) [improve][build] Upgrade Gradle to 9.8.2 (#27148) [improve][broker] Make entry cache copying dynamically configurable (#27144) [fix][admin] Fix Pulsar Admin client deadlock caused by AsyncHttpClient’s per-host connection limit (#27022) [fix][broker] Prevent dispatch rate limiting overshoot with batched messages (#27141) [improve][meta] Upgrade Oxia Java client to 0.10.3 (#27131) [improve][build] Replace grpc-netty-shaded with grpc-netty (#27128) [fix][broker] Return errors for discarded ACKs with receipts (#27103) [improve][doc] Document TopicName#isPartitioned and clarify getPartitionedTopicName (#27119) [improve][cli] Show the compression type when peeking or examining messages (#27100) [improve][cli] Support compacted reads in client commands (#27059) [fix][broker] Clear pending read flags when a read fails before the cursor takes ownership (#27125) [improve][broker] Balance partitions across consumers with activeConsumerFailoverConsistentHashing (#27107) [fix][ml] Fix ManagedCursor missing active-cursor registration during concurrent state changes (#27102) [fix][client] Fail creation on explicit TLS client certificate alerts (#27063) [fix][client] Fix new topics and partitions not receiving messages after resume (#27124) [improve][meta] Upgrade Oxia Java client to 0.10.2 (#22749) [fix][client][txn] Fix client connection closed before using (#27098) [fix][client] Don’t clear shared broker entry metadata when recycling a pooled message (#27109) [fix][broker] Fix Key_Shared replay hash count races during mark-delete cleanup (#27122) [fix][test] Fix flaky ZKMetadataStoreBatchIOperationTest.testReceivedHugeResponse (#27113) [cleanup][misc] Fix the remaining javac warnings in main sources

AoP

671c9e5 Fix OpenTelemetry SPI version alignment with Pulsar BOM [fix] Resume consumer after netty channel writable change

MoP

5b2efc61 Fix MQTT stats tests with Gson map deserialization changes

KoP

[branch-5.0] Bump Pulsar BOM to 5.0.0.3 and align build and test dependencies

StreamNative Pulsar Plugins

fix(detector): upgrade go-jose to v4.1.6 for security fixes

pulsarctl

fix: install Trivy directly from GitHub Releases

Cloud Pulsar Plugins

dc180b55 [CI] Fix free-disk-space action repository reference [Rest API Interceptor] Export billing resource usage [Rest API Interceptor] Always report resource usage metrics

StreamNative Unified RBAC

Bump the RBAC SDKs to 1.17.3 (5.0)

Security Fixes

Apache Pulsar

(#27343) [fix][sec] Upgrade ZooKeeper to 3.9.7 to address several CVEs (#27310) [fix][sec] Upgrade Netty to 4.2.20 and netty-tcnative to 2.0.85 to address several CVEs and bugs (#27186) [fix][sec] Upgrade zstd-jni to 1.5.7-25 (#27191) [fix][sec] Upgrade OkHttp to 5.5.1 and Okio to 3.18.2 (#27192) [fix][sec] Upgrade Gson to 2.15.0 (#27188) [fix][sec] Upgrade OpenTelemetry to 1.67.0 and OpenTelemetry instrumentation to 2.33.1 (#27193) [fix][sec] Upgrade Byte Buddy to 1.18.16 (#27190) [fix][sec] Upgrade commons-codec to 1.23.0 (#27187) [fix][sec] Upgrade lz4-java to 1.12.1 (#27189) [fix][sec] Upgrade JLine to 4.4.8 (#27185) [fix][sec] Upgrade java-jwt to 4.6.2 and fail OIDC authentication on out-of-range timestamps (#27184) [fix][sec] Upgrade snappy-java to 1.1.10.12 (#27183) [fix][sec] Upgrade Vert.x to 4.5.35 (#27178) [fix][sec] Upgrade Checkstyle to 14.3.0 and use Guava 33.7.2-jre on its classpath (#27181) [fix][sec] Bump Jackson to 2.21.8 (#27156) [fix][sec] Bump guava to 33.7.2-jre (#27142) [fix][sec] Bump github.com/go-jose/go-jose/v4 from 4.1.4 to 4.1.6 in /pulsar-function-go (#27139) [fix][sec] Bump org.xerial.snappy:snappy-java from 1.1.10.8 to 1.1.10.9 (#27138) [fix][sec] Use guava 33.7.1-jre also in jclouds-shaded