What’s Changed
🎉 Major New Features
Compaction Scheduler Enhancements (Major Enhancement)
- Resource Configuration: Full support for compaction scheduler resource and JVMOptions configuration
- Label Management: Fixed compaction scheduler labels for better resource identification
- OpenTelemetry Integration: Support for OTel JVM parameters with proper spacing
- Multi-Backend Storage: Enhanced support for S3, Local, GCS, AzureBlob, and AzureDfs storage backends
- Cloud Storage Config: New cloud storage configuration options
- Local Cache Support: Improved local cache configuration with storage path and size management
- Catalog Integration: Support for multiple catalogs and catalog entities
- Deployment Modes: Support for both StatefulSet and Deployment deployment modes
RBAC Security Enhancements (Major Security Improvement)
- Version-Aware RBAC: RBAC filter only enabled when broker version >= 3.3.2.5 or >= 4.0.0.9
- RBAC List Filter: New rbac-list-filter interceptor enabled when RBAC is enabled
- Enhanced Authorization: Improved role-based access control mechanisms
- Security Context: Better security context management for all components
Ursa Integration Improvements (Enhanced Cloud Storage)
- Kafka Connect Support: Added Ursa Kafka Connect integration
- Istio Proxy Configuration: Added CPU request and limit for istio-proxy in Ursa clusters
- Backend Storage: Disabled BookKeeper backed package service for Ursa clusters
- Storage Configuration: Enhanced Ursa storage configuration options
Batch Processing Enhancements (Performance Improvement)
- Batch Index Ack: Enabled batch index acknowledgment by default for better performance
- Batch Processing: Improved batch processing capabilities across components
🔧 Feature Improvements
Graceful Rollout System Enhancements
- Servlet Support: Added servlet for graceful rollout operations
- Revision Management: Improved revision deletion logic for graceful rollout
- Function Owner Fix: Fixed function owner assignment when graceful rollout is enabled
- Scale Down Logic: Enhanced scale down revision when old pods are crashing
Oxia Cluster Improvements
- Namespace Reconciliation: Improved Oxia namespace reconciliation logic
- Volume Name Fix: Fixed Oxia client volume name length limitation (max 63 characters)
- Istio Integration: Enhanced Istio gateway, virtual service, and service entry deployment
- Configuration Management: Better configuration metadata store config mount
- Client Authentication: Support for passing auth config for broker Oxia client
Function Mesh Integration
- Functions Worker Mount: Functions worker YAML now mounts to broker even when function is disabled
- Role and Role Binding: Create role and role binding for catalog without function feature flag
- Function Management: Improved function lifecycle management
PFSQL Integration
- Version Upgrade: Bumped PFSQL to version 0.20.0
- Enhanced Integration: Better integration with Pulsar ecosystem
Detector and Monitoring
- Conditional Deployment: Detector deployment only enabled when brokers are initialized and replicas > 0
- Resource Optimization: Better resource utilization for monitoring components
🐛 Bug Fixes
Controller and Reconciliation Fixes
- Pod Watching: Fixed pod watching in PulsarBroker controller
- Broker Ready Status: Fixed broker ready status detection
- Cleanup Logic: Fixed cleanup replication cluster retry logic and NPE issues
- TLS Configuration: Fixed TLS configuration issues
- Metadata Store: Updated clear metadata store check logic
Graceful Rollout Fixes
- Event Source: Fixed graceful rollout event source issues
- Revision Management: Fixed graceful rollout revision deletion
- Function Ownership: Fixed function owner assignment during graceful rollout
Oxia Cluster Fixes
- Compatibility: Fixed Oxia compatibility issues
- Volume Naming: Fixed Oxia client volume name length issues
- Configuration: Fixed Oxia configuration metadata store config mount
Istio Integration Fixes
- HTTPS Protocol: Fixed Istio HTTPS protocol handling
- Service Entry: Fixed ZooKeeper service entry update issues
- Mesh Detection: Removed on-mesh check and updated TLS protocol
Security and Authentication Fixes
- JWT Token Expiry: Fixed max expiry time for JWT tokens
- Token Management: Specify expiry for non-expiring tokens
- API Keys: Fixed API key secret name when graceful rollout is enabled
🚀 Performance Optimizations
Batch Processing
- Default Batch Index Ack: Enabled by default for improved performance
- Batch Operations: Enhanced batch processing across all components
Resource Management
- CPU Limits: Added CPU request and limit for istio-proxy in Ursa clusters
- Resource Optimization: Better resource allocation and management
Storage Optimizations
- Compaction Scheduler: Improved compaction scheduler resource configuration
- Local Cache: Enhanced local cache management for better performance
🔄 Dependencies and Infrastructure
Security Updates
- Crypto Upgrade: Upgraded crypto libraries to fix CVE vulnerabilities
- Base Image Updates: Updated base images for security improvements
CI/CD Improvements
- Kubernetes Version Matrix: Added CI k8s version matrix testing
- Release Configuration: Updated OLM release configurations
- Manifest Fixes: Fixed operator manifests
External Dependencies
- Trivy Database: Using public ECR registry for downloading Trivy databases
- Function Mesh: Updated function mesh integration
📋 Configuration Changes
New Configuration Options
- Compaction Scheduler: New resource and JVM options configuration
- RBAC Filter: Version-aware RBAC filter configuration
- Batch Index Ack: Default enabled batch index acknowledgment
- Ursa Storage: Enhanced Ursa storage configuration options
Feature Flags
- RBAC List Filter: New RBAC list filter interceptor
- Compaction Scheduler: Enhanced compaction scheduler configuration
- Ursa Integration: Improved Ursa cluster integration
⚠️ Breaking Changes
- RBAC Filter: RBAC filter now only enabled for specific broker versions (>= 3.3.2.5 or >= 4.0.0.9)
- Compaction Scheduler: New resource configuration requirements
- Oxia Volume Names: Volume names must be 63 characters or less
- Function Worker Mount: Functions worker YAML now mounts even when function is disabled
📚 Documentation and Examples
- Updated compaction scheduler configuration examples
- Enhanced Ursa integration documentation
- Improved RBAC configuration guides
- Updated graceful rollout documentation