> ## Documentation Index
> Fetch the complete documentation index at: https://docs.streamnative.io/llms.txt
> Use this file to discover all available pages before exploring further.

# V4.1.0.4

# StreamNative Weekly Release Notes v4.1.0.4

## Download

### Distributions

* [https://github.com/streamnative/pulsar/releases/tag/v4.1.0.4](https://github.com/streamnative/pulsar/releases/tag/v4.1.0.4)

### Packages

* [Maven Central](https://search.maven.org/artifact/io.streamnative/pulsar/4.1.0.4/pom)

### Images

* [sn-platform](https://hub.docker.com/layers/streamnative/sn-platform/4.1.0.4/images/sha256-c49aecb7969caa30e419aff3e912004555aa8e6484ebe287980c5168df681e58)

* [sn-platform-slim](https://hub.docker.com/layers/streamnative/sn-platform-slim/4.1.0.4/images/sha256-cfbadf1830b4d9c1675050d3162e5900f1f5f5f0f7d301eb7096799e05189ef8)

* [private-cloud](https://hub.docker.com/layers/streamnative/private-cloud/4.1.0.4/images/sha256-cfbadf1830b4d9c1675050d3162e5900f1f5f5f0f7d301eb7096799e05189ef8)

## General Changes

### Apache Pulsar

([#24863](https://github.com/apache/pulsar/pull/24863)) \[fix]Fixed getChildren('/') on Oxia based provider

([#24852](https://github.com/apache/pulsar/pull/24852)) \[fix]\[ml] Fix `getNumberOfEntries` may point to deleted ledger

([#24855](https://github.com/apache/pulsar/pull/24855)) \[fix]\[ml] Fix ledger trimming race causing cursor to point to deleted ledgers

([#24838](https://github.com/apache/pulsar/pull/24838)) \[fix]\[broker] Ensure LoadSheddingTask is scheduled after metadata service is available again

([#24841](https://github.com/apache/pulsar/pull/24841)) \[improve]\[ci] Upgrade GitHub Actions workflows to use ubuntu-24.04

([#24840](https://github.com/apache/pulsar/pull/24840)) \[improve] Upgrade Alpine base image to 3.22 version

([#24836](https://github.com/apache/pulsar/pull/24836)) \[fix]\[ml] PIP-430: Fix concurrency issue in MessageMetadata caching and improve caching

([#24830](https://github.com/apache/pulsar/pull/24830)) \[fix]\[client] Fix getPendingQueueSize for PartitionedTopicProducerStatsRecorderImpl: avoid NPE and implement aggregation

([#24832](https://github.com/apache/pulsar/pull/24832)) \[fix] Fix mixed lookup/partition metadata requests causing reliability issues and incorrect responses

([#24821](https://github.com/apache/pulsar/pull/24821)) \[fix]\[test] Fixed nondeterministic JSON ordering in multiple tests

([#24798](https://github.com/apache/pulsar/pull/24798)) \[improve]\[client] PIP-420: Update the schema ID format

([#24825](https://github.com/apache/pulsar/pull/24825)) \[improve]\[broker] Cache last publish timestamp for idle topics to reduce storage reads

([#24815](https://github.com/apache/pulsar/pull/24815)) \[feat]\[monitor] Add ML write latency histogram and entry size histogram as OTel metrics

([#24829](https://github.com/apache/pulsar/pull/24829)) \[fix]\[broker] Allow intermittent error from topic policies service when loading topics

([#24822](https://github.com/apache/pulsar/pull/24822)) \[fix]\[client] Make auto partitions update work for old brokers without PIP-344

([#24810](https://github.com/apache/pulsar/pull/24810)) \[feat]\[monitor] Add publish latency histogram as OTel metrics

([#24823](https://github.com/apache/pulsar/pull/24823)) \[fix]\[test] Fix flaky SingleThreadNonConcurrentFixedRateSchedulerTest.testPeriodicTaskCancellation

([#24824](https://github.com/apache/pulsar/pull/24824)) \[improve]\[ml] Upgrade Oxia client to 0.7.0

([#24790](https://github.com/apache/pulsar/pull/24790)) \[feat]\[client] Implement PIP-234 for sharing thread pools and DNS resolver/cache across multiple Pulsar Client instances

([#24734](https://github.com/apache/pulsar/pull/24734)) \[fix]\[client] Fix PulsarAdmin description check and add test

([#24729](https://github.com/apache/pulsar/pull/24729)) \[improve]\[client] Allow adding custom description to User-Agent header

([#24728](https://github.com/apache/pulsar/pull/24728)) \[fix]\[client] Add description method to ClientBuilder

([#24812](https://github.com/apache/pulsar/pull/24812)) \[fix]\[build] Remove invalid profile in settings.xml that caused gpg signing to fail

([#24811](https://github.com/apache/pulsar/pull/24811)) \[fix]\[build] Fix maven deploy with maven-source-plugin 3.3.1

([#24801](https://github.com/apache/pulsar/pull/24801)) \[improve]\[broker]Improve NamespaceService log that is printed when cluster was removed

([#24785](https://github.com/apache/pulsar/pull/24785)) \[fix]\[broker] Fix incorrect topic loading latency metric and timeout might not be respected

([#24770](https://github.com/apache/pulsar/pull/24770)) \[fix]\[broker] Flaky-test: ExtensibleLoadManagerImplTest.testDisableBroker

([#24784](https://github.com/apache/pulsar/pull/24784)) \[improve]\[client/broker] Add DnsResolverGroup to share DNS cache across multiple PulsarClient instances

([#24780](https://github.com/apache/pulsar/pull/24780)) \[improve]\[broker] Replace isServiceUnitActiveAsync with checkTopicNsOwnership

([#24813](https://github.com/apache/pulsar/pull/24813)) \[fix] Update gRPC to 1.75.0

### AoP

74655b2 Use commons-lang3

### MoP

a57d06db update pulsar and sn dependency version

### KoP

Handle invalid topic format error for metadata request

feat(kafka-admin): Support describing topic configs with `kop.kafka.` prefix

Use the Pulsar format for the Kafka consumer offsets topic

fix(kafka-impl): correct topic partition extraction from Pulsar topic names

Reduce spamming logs from schema registry and topic lookup

\[Ursa] Fix the wrong position comparison in topic replay

492502d4a Bump version to 4.1.0.4

Add producer ID expiration mechanism for classic engine

Fix released produce request buffer could be accessed

Make avro-maven-plugin version consistent with avro

Use new APIs from Oxia 0.7.0

Write transaction log and offset log with partition log

\[schema-registry] add option to disable compatibility mode configuration

Add more info to logs when the connection is closed

Fix direct memory oom with ack=0

\[Ursa] Improve producer state snapshot taking to avoid metadata thread

Refactor schema provider to use Confluent avro schema provider

\[Ursa] Remove oldest producers when the serialized producer state is too large

Support Pulsar and Kafka client communicate with the Kafka schema registry

Support parse googles built in proto files on protobuf schema

Fix idempotent producer for classic engine

Improve topic loading time by skipping Pulsar message deduplication recovery

Add test cases for record schema validation with subject name strategy

### StreamNative Pulsar Plugins

Upgrade commons-lang to commons-lang3

Upgrade Oxia to 0.7.0

Upgrade zk version to 3.9.4 to fix CVE

### Cloud Pulsar Plugins

5fd37f2 Use commons-lang3

### Function Mesh Worker Service

Use sn-operator to deploy pulsar cluster in CI

fix agent cannot update some fields error

Fix ci failure

fix build and license header

### StreamNative Tiered storage

Use lang3 package StringUtils to replace lang package StringUtils

Upgrade aws sdk to 2.32.28 to keep sync with Pulsar

### StreamNative Ursa storage

Respect markDeletedOffsets when reading entries from PersistStorageApi

Fix pulsar expired ledger not deleted bug

Carry the number of messages info for positions in Entry

Fix delta not support timestamp\_ntz

Do iceberg schema compatibility check before updating schema

Use strict match rule to fetch token from UnityCatalogSasTokenProviderTest

Disable flaky AsyncCleanerTest#simpleTest

Fix parquet reader handle union type bug

fix: support nested field partitioning in Iceberg tables

Remove METADATA\_DELETE\_AFTER\_COMMIT\_ENABLED property

Skip remove preserved properties

Fix catalog close bug

Add more log for parse entry

Move the schema evolution logic into the iceberg table

Upgrade Oxia client to 0.7.0

Bump org.apache.commons:commons-lang3 from 3.17.0 to 3.18.0

Add more admin commands

Abstract the schema evolution parts in the encoder

Removed dependency on commons-lang 2.6

## Security Fixes