- References
- StreamNative Pulsar Changelogs
- v2.11
StreamNative Weekly Release Notes v2.11.1.0
StreamNative Weekly Release Notes v2.11.1.0
Download
Distributions
- https://github.com/streamnative/pulsar/releases/tag/v2.11.1.0
Packages
Images
General Changes
Apache Pulsar
[fix] [ml] Fix uncompleted future when remove cursor
[refactor][broker] Use AuthenticationParameters for rest producer
[fix] [cli] Fix Broker crashed by too much memory usage of pulsar tools
[fix][proxy] Fix connection read timeout handling in Pulsar Proxy
[improve][proxy] Implement graceful shutdown for Pulsar Proxy
[fix][build] Suppress Guava CVE-2020-8908 in OWASP dependency check
[fix][build] Client modules should be built with Java 8
[fix][broker] Only validate superuser access if authz enabled
[improve][txn] Cleanup how superusers abort txns
[refactor][fn] Use AuthorizationServer more in Function Worker API
[fix] [admin] fix incorrect state replication.connected on API partitioned-topic stat
[fix][broker] Return if AbstractDispatcherSingleActiveConsumer closed
[fix][client] Fix DeadLetterProducer creation callback blocking client io thread.
[fix] [admin] Make response code to 400 instead of 500 when delete topic fails due to enabled geo-replication
[fix][broker] Fix NPE when update topic policy.
[fix][meta] Follow up #19817, Fix race condition between ResourceLock update and invalidation
[fix][meta] Follow up #19817, Fix race condition between ResourceLock update and invalidation
[improve][broker] Follow up #19230 to tighten the validation scope (#19234)
[improve][broker] Authorize originalPrincipal when provided
[fix] [ml] make the result of delete cursor is success if cursor is deleted
[fix][meta] Fixed race condition between ResourceLock update and invalidation
[improve] Upgrade bookkeeper to 4.15.4
[fix][broker] Copy subscription properties during updating the topic partition number.
[fix][meta] Fix close borrowed executor
[fix][broker] Fix issue where msgRateExpired may not refresh forever
[fix][meta] Fix deadlock causes session notification not to work
[fix][broker] Fix potential exception cause the policy service init fail.
[fix] [broker] delete topic failed if disabled system topic
[fix][client] Fix topic list watcher fail log
[fix][broker] Fix index generator is not rollback after entries are failed added.
[improve][cli]Upgrade python to python3
[fix][cli] Fix Pulsar admin tool is ignoring tls-trust-cert path arg
[fix] [broker] Topic close failure leaves subscription in a permanent fence state
[fix][broker][branch-2.11] Fix geo-replication admin
[improve] Simplify enabling Broker, WS Proxy hostname verification
[fix][broker] Filter system topic when getting topic list by binary proto.
[feat][txn] Support for idempotent commit and abort, Solution1.
[fix][ci][branch-2.11] Fix broken CI
[fix][client] Broker address resolution wrong if connect through a multi-dns names proxy
[fix][test] ProxyWithAuthorizationTest remove SAN from test certs
[improve][broker][branch-2.11] Improve tls config on replication client and cluster cli
[fix] [client] fix memory leak if enabled pooled messages
[fix][txn]fix receive duplicated messages due to pendingAcks in PendingAckHandle
[fix][broker] Fixed history load not releasing
[fix][broker] Allow proxy to pass same role for authRole and originalRole
[fix][broker] Fix geo-replication admin
[fix] [test] Wrong mock-fail of the test ManagedLedgerErrorsTest.recoverLongTimeAfterMultipleWriteErrors
[fix][broker] Terminate the async call chain when the condition isn't met for resetCursor
[fix][proxy] Fix using wrong client version in pulsar proxy
[improve][broker] Use shrink map for trackerCache
[fix] [broker] Make the service name resolver cache of PulsarWebResource expire after access
[fix][authentication] Store the original authentication data
[fix][broker] Copy command fields and fix potential thread-safety in ServerCnx
[fix][client] Fix authentication not update after changing the serviceUrl
[fix][broker] Make ServerCnx#originalAuthData volatile
[fix][broker] Make authentication refresh threadsafe
[fix] [broker] Incorrect service name selection logic
[fix][proxy] Fix JKS TLS transport
[fix][client] Fix load the trust store file
[fix][fn] Fix k8s merge runtime opts bug
[improve][txn] Allow superusers to abort transactions
[improve][broker] Require authRole is proxyRole to set originalPrincipal
[fix][broker] Expect msgs after server initiated CloseProducer
[fix] [ml] topic load fail by ledger lost
[revert][misc] "modify check waitingForPingResponse with volatile (#12615)"
[improve][broker] Add test to verify authRole cannot change
[fix][ml] Reset individualDeletedMessagesSerializedSize after acked all messages.
[improve] Upgrade lombok to 1.8.26
[improve] Upgrade to zk 3.8.1
[fix][broker] Fix loadbalance score calculation problem
[improve][misc] Upgrade Netty to 4.1.87.Final
[fix][ml] Fix potential NPE cause future never complete.
[fix][authorization] Fix the return value of canConsumeAsync
[improve] [broker] Print warn log if compaction failure
[fix] [ml] Fix the incorrect total size if use ML interceptor
[fix][broker] Replace sync call in async call chain of AdminResource#internalCreatePartitionedTopic
[fix][broker] Release Netty buffer in finally block in ServerCnx#handleLastMessageIdFromCompactedLedger
[fix][client] Fix async completion in ConsumerImpl#processPossibleToDLQ
[fix][broker] Replace sync method in NamespacesBase#internalDeleteNamespaceBundleAsync
[fix][broker] Fix concurrency bug in PersistentTopicsBase#internalGetReplicatedSubscriptionStatus
[fix] Close TransactionBuffer when create persistent topic timeout
[fix][io] Update Elasticsearch sink idle cnx timeout to 30s
[fix] [ml] messagesConsumedCounter of NonDurableCursor was initialized incorrectly
[fix][client] Fix reader listener can't auto ack with pooled message.
[fix] [ml] The atomicity of multiple fields of ml is broken
[improve][broker] Added isActive in ManagedCursorImpl
[improve] Upgrade wildfly-elytron (used by debezium) to fix CVE-2022-3143
[fix][proxy] Only go to connecting state once
[fix][client] Set fields earlier for correct ClientCnx initialization
[improve][broker] ServerCnx: go to Failed state when auth fails
[improve][txn] Handle changeToReadyState failure correctly in TC client
[fix][broker] AbstractBatchedMetadataStore - use AlreadyClosedException instead of IllegalStateException
[fix] [ml] Topics stats shows msgBacklog but there reality no backlog
[cleanup][broker] Validate originalPrincipal earlier in ServerCnx
[fix][sec] Exclude log4j from openmldb
[fix][txn] Catch and log runtime exceptions in async operations
[fix] [broker] Counter of pending send messages in Replicator incorrect if schema future not complete
[improve][broker] Follow up #19230 to tighten the validation scope
[fix][broker] Support deleting partitioned topics with the keyword -partition-
[fix][broker] Copy subscription properties during updating the topic partition number.
[improve][websocket] Add ping support
[fix][doc] Remove lombok plugin and define FunctionRecordBuilder explicitly
[fix][txn] fix txn coordinator recover handle committing and aborting txn race condition.
[fix][broker] Fix race condition while updating partition number
[fix][client] Fix producer could send timeout when enable batching
[fix][broker] Pass subName for subscription operations in ServerCnx
[fix][build] Upgrade dependency-check-maven plugin to fix broken OWASP check
[improve][broker] Add ref count for sticky hash to optimize the performance of Key_Shared subscription
[fix][broker] Close transactionBuffer after MessageDeduplication#checkStatus failed
[fix][broker] Support zookeeper read-only config.
[fix][txn] Always send correct transaction id in end txn response
[improve][schema] Do not print error log with stacktrace for 404
[fix][broker] Topic could be in fenced state forever if deletion fails
[fix][broker] Fix consumer with schema cannot subscribe when AUTO_CONSUME consumers exist
[fix][build] Resolve OWASP Dependency Check false positives
[improve][sec] Suppress false positive OWASP reports
[fix][sec] Upgrade woodstox to 5.4.0
[fix][sec] Upgrade jettison to 1.5.3
[fix][client] Prevent DNS reverse lookup when physical address is an IP address
[improve][admin] Improve partitioned-topic condition evaluation
[improve][broker] Add logs for why namespace bundle been split
[fix][admin] Fix validatePersistencePolicies
that Namespace/Topic persistent policies cannot set to < 0
[improve][broker] Omit making a copy of CommandAck when there are no broker interceptors
[fix][broker] Fix deadlock in PendingAckHandleImpl
[fix][broker] Copy proto command fields into final variables in ServerCnx
[fix][broker]fix multi invocation for ledger createComplete
[fix] [tx] [branch-2.11] Transaction buffer recover blocked by readNext
[improve] Add PulsarExceptionBase class that supports slf4j like parameterized string formatting
[fix][txn] transaction pending ack store future not completely problem
[fix][broker]Update interceptor handler exception
[improve][sec] Remove snakeyaml dependency to suppress CVE-2022-1471
[fix][broker] Check operstate
when get physical NICs
[fix] [broker] getLastMessageId returns a wrong batch index of last message if enabled read compacted
[fix][fn] Typo in method name
[fix][test][branch-2.10] Remove testGetTopic method
[cherry-pick][branch-2.11] cherry-pick fixing can not delete namespace by force (#18307)
[fix][broker] Fix uncompleted future when getting the topic policies of a deleted topic
[fix][broker] Fix delete system topic clean topic policy
[fix][broker] Fix namespace deletion if __change_events topic has not been created yet
[fix][sql] Fix message without schema issue.
[cleanup][broker] Simplify extract entryMetadata code in filterEntriesForConsumer
[fix][broker] Fix duplicated schemas creation
[fix][broker] transactional producer created failed due to read snapshot blocked.
[fix][broker] Fix PulsarRegistrationClient and ZkRegistrationClient not aware rack info problem.
[improve][test] Add test brokerReachThreshold
for ThresholdShedderTest
[improve][broker] Make Consumer#equals more effective
[fix][broker] Create replicated subscriptions for new partitions when needed
[fix][client] For exclusive subscriptions, if two consumers are created repeatedly, the second consumer will block
[fix][io] ElasticSearch sink: align null fields behaviour
[improve][broker] Adjust to sample the error log
[fix][broker] DnsResolverUtil.TTL should be greater than zero
[fix][client] Fix multi-topic consumer stuck after redeliver messages
[fix][client] Avoid redelivering duplicated messages when batching is enabled
[fix][client] Fix failover/exclusive consumer with batch cumulate ack issue.
[fix][broker] Correctly set byte and message out totals per subscription
[improve][client] Change the get lastMessageId to debug level
[fix][broker] Fix open cursor with null-initialPosition result with earliest position
[fix][broker]Fix failover/exclusive consumer cumulate ack not remove msg from UnAckedMessageTracker
[fix][client] Fix possible npe
[fix][client] Fixes batch_size not checked in MessageId#fromByteArrayWithTopic
[fix][client] Fix the Windows absolute path not recognized in auth param string
[fix][client] Set authentication when using loadConf in client and admin client
[fix][client] Fix exception when calling loadConf on a ConsumerBuilder that has a KeySharedPolicy
[fix][client] Support LocalDateTime Conversion
[fix][broker] Fix can not delete namespace by force
[fix][broker] In the trimDeletedEntries method, release the removed entry
[fix][schema] Fix creating lots of versions for the same schema
[fix][broker] fix delete_when_subscriptions_caught_up doesn't work while have active consumers
[fix][client] Fix IllegalThreadStateException
when using newThread in ExecutorProvider.ExtendedThreadFactory
[fix] [pulsar-client] Fix pendingLookupRequestSemaphore leak when Ser…
[improve][broker] Add UncaughtExceptionHandler for every thread pool
[improve][broker] invalidate lock cache when Deleted NotificationType
[improve][broker] Remove locallyAcquiredLock when removeOwnership
[improve][broker] Support setting forceDeleteTenantAllowed
dynamically
[improve][broker] Support setting ForceDeleteNamespaceAllowed
dynamically
[fix][meta] fix getChildren
in MemoryMetadataStore and EtcdMetadataStore
[fix][broker] AVAILABLE_PERMITS_UPDATER not updated, when writeAndFlush fails.
[fix][broker] PulsarRegistrationClient - implement getAllBookies and follow BookieServiceInfo updates
[fix][broker] Fix update authentication data
[fix][function] Fix invalid metric type "gauge "
[fix][fn] fix function failed to start if no typeClassName
provided in FunctionDetails
[improve][client] Support MAX_ACK_GROUP_SIZE configurable
[improve][schema] Change update schema auth from tenant to produce
[fix][broker] Update the log print content of createSubscriptions
[improve][broker] Reduce unnecessary persistence of markdelete
[fix][broker] Fix the order of resource close in the InMemoryDelayedDeliveryTracker
[fix][broker] Fix incorrect bundle split count metric
[fix][broker]unify time unit at dropping the backlog on a topic
[improve][connector] JDBC sink: allow any jdbc driver
Make BookieId work with PulsarRegistrationDriver (second take)
[fix][broker]Fix mutex never released when trimming
Allow to configure and disable the size of lookahead for detecting fixed delays in messages
[fix][admin] returns 4xx error when pulsar-worker-service is disabled and trying to access it
[fix][loadbalance] Fix the wrong NIC speed rate unit.
[improve][broker]Improve PersistentMessageExpiryMonitor expire speed when ledger not existed
[fix][client] moving get sequenceId into the sync code segment
[refactor][java] Unify the acknowledge process for batch and non-batch message IDs
[fix][proxy] Fix refresh client auth
[fix][cli] Check numMessages after incrementing counter
[improve][java-client]Add init capacity for messages in BatchMessageContainerImpl
[fix][doc]fix comments for exposeManagedLedgerMetricsInPrometheus field
[Improve][auth]Update authentication failed metrics report
[fix][sec] Bump snakeyaml to 1.32 for CVE-2022-38752
[fix][cli] Quit PerformanceConsumer after receiving numMessages messages
[bugfix] ManagedLedger: move to FENCED state in case of BadVersionException
[fix][broker] Fix create ns
[fix][metrics]wrong metrics text generated when label_cluster specified
[fix][broker] Fix executeWithRetry result is null
[fix][cli] Fix mbeans to json
[fix][bookie] Correctly handle list configuration values
[fix][build] duplicate entry when merging services
[C++] Reset havePendingPingRequest
flag for any data received from broker
[improve][pulsar-proxy] Update proxy lookup throw exception type
Issue 17588: Allow deletion of a namespace that was left in deleted status
[improve][test] Remove WhiteBox for ElasticSearchSinkTests
[improve][test] remove WhiteBox on MockZooKeeper
[improve][ci] Remove post commit builds and add manual workflows trigger
[ci] Move more tests to the flaky suite
[ci] Skip run flaky tests suite for cpp-only or doc-only changes
[ci] Extract FLAKY tests suite to a dedicated workflow
[ci] Reduce runners load for pulls that affect only doc or cpp changes
[fix][tiered-storage] Don't cleanup data when offload met Metastore exception
[improve][ci] Optimize "Pulsar Bot" workflow
[improve][ci] Remove "Cancel duplicate workflows" workflow
[fix][flaky-test]NamespaceServiceTest.flaky/testModularLoadManagerRemoveBundleAndLoad
[fix][license] Update the log4j version in the presto license file
[fix][sec] bump snakeyaml to 1.31 fix CVE-2022-25857
[improve][broker] Improve cursor.getNumberOfEntries if isUnackedRangesOpenCacheSetEnabled=true
[fix][client]Duplicate messages when use MultiTopicsConsumerImpl
[improve][cli] Using separate TLS config on the compactor
[improve][txn] Add getState in transaction for client API
[fix][broker] Remove timestamp from Prometheus metrics
[improve][broker] Using handle
instead of handleAsync
to avoid using common pool thread
[fix][broker] fix can not revoke permission after update topic partition
[improve][admin] PulsarAdminBuilderImpl overrides timeout properties passed through config map
[fix][broker]fix catching ConflictException when update topic partition
[fix][broker] Fix Npe thrown by splitBundle
[fix][flaky] Fix flakyness in testAutoSchemaFunction
[improve][broker]Only create extended partitions when updating partition number
[improve][schema]autoSkipNonRecoverableData work for schema Ledger
[improve][ci] Replace test reporting with less verbose solution
[improve][broker]Recover as much cursor data as possible, even if entry is invalid
[fix][broker]ManagedLedger metrics fail cause of zero period
[refactor][java] Improve docs and code quality about KeyValueSchema usages
[fix][broker]fail to update partition meta of topic due to ConflictException: subscription already exists for topic
[broker][fix]Fix update topic remove properties
[improve][build] Avoid building image multiple times
[fix][broker]fix arithmetic exception for LeastResourceUsageWithWeight strategy
[improve][docker] Switch to Temurin JDK
[fix][broker] Skip not connectable URL when get redirectionUrl
[improve][admin] Unset namespace policy to improve deleting namespace.
[fix][broker] Fix delivery the message that has been acknowledged
[fix][txn] Correct the prompt message
[improve][ci] Skip unnecessary tests when there are only cpp/python related changes
[Fix][flaky-test] Fix testConsumeTxnMessage
[Improve][broker]Reduce GetReplicatedSubscriptionStatus local REST call
[fix][ml] fix NPE
[improve] clean the empty topicAuthenticationMap in zk when revoke permission
[pulsar-io] KCA: handle kafka's logical schemas
[improve][broker] ServerCnx: log at warning level when topic not found
[fix][storage] Autorecovery default reppDnsResolverClass to ZkBookieRackAffinityMapping
[fix][broker][functions-worker] Ensure prometheus metrics are grouped by type (#8407, #13865)
[refactor][broker] Arrange cleanup operation for topic creation
[monitor][txn] Add metrics for transaction
Issue 14583: [python client]: Improve garbage collection of producer/consumer/reader objects
[feature][txn] Fix individual ack batch message with transaction abort redevlier duplicate messages
[Improvement] Recycler should use io.netty.recycler.maxCapacityPerThread instead of io.netty.recycler.maxCapacity.default as maxCapacityPerThread configuration in Netty 4.1.x
[Issue 12844][pulsar-client-cpp] Excessive locking cause significant performance degradation
KoP
Increase timeout for KafkaListenerNameTest
[Schema] Fix wrong response when querying schema by subject and version
[bugfix] Fix regression with the upgrade to 2.8.x: Count corrently the number of partitions in ProduceRequest
[Schema Registry] Fix the delete operation not work
[Schema] Return 401 error when no HTTP authentication is configured
Return a correct metadata response when topic is not found
[test] Add kafka admin api unit test
Improve getting started guide
[fix][schema] Return JSON error message for failed request
pulsarctl
Fix vulnerabilities
Function Mesh Worker Service
fix ci in branch 2.11
fix build branch 2.11
adopt new api branch 2.10.3
Bump function-mesh to v0.13.0
support start\stop\restar function\sink\source