What’s Changed
🎉 Major New Features
UniLink Data Migration Platform (Brand New Feature)
- UniLink CRD: Complete support for Kafka to Pulsar data migration with UniLink
- UniLinkSchema CRD: Schema registry integration for seamless data migration
- Subject Mappings: Support for flexible subject mappings during migration
- Bring Your Own Bucket: Support for custom bucket configurations in UniLink
- Metrics Collection: Built-in metrics collection and monitoring for UniLink
- Schema Registry Integration: Full integration with source schema registries
- Topic Mappings: Flexible topic mapping configurations for data migration
- Consumer Group Management: Complete consumer group configuration support
Shadow Namespace Load Manager (Major Enhancement)
- Shadow Namespace Support: New load manager type for optimized namespace handling
- Kafka-on-Pulsar Integration: Specialized load manager for KoP (Kafka on Pulsar) workloads
- Bundle Management: Optimized namespace bundle configuration (64 bundles by default)
- Load Balancing: Disabled shedding and auto-split for shadow namespace workloads
- Performance Optimization: Specialized configuration for high-throughput scenarios
Enhanced Gateway and Networking (Major Improvement)
- Topology-Aware Gateway: Support for additional selector configuration
- Multiple Gateway Support: Fixed authentication policy ports for multiple gateways
- Istio Integration: Enhanced Istio integration with proper hostname handling
- RBAC Proxy Removal: Removed kube-rbac-proxy dependency for simplified architecture
- Port Management: Excluded kube-rbac-proxy port (8443) from Istio sidecar
Advanced Function Management (New Feature)
- Function Download Control: Support for enableFunctionDownload configuration
- Trusted Mode: Support for enableTrustedMode on function workers
- Enhanced Security: Better function execution security controls
- Resource Management: Improved function resource allocation
🔧 Feature Improvements
Data Lake and Catalog Enhancements
- S3 Table REST API: Support for S3 table REST API catalog
- Memory Compute Fix: Fixed memory compute issues in S3 table configurations
- REST Catalog Config: Fixed S3 table REST catalog configuration
- Region Support: Enhanced region configuration for S3 table REST API
Graceful Rollout Improvements
- Hash Fallback Logic: Added fallback logic for graceful rollout hash calculation
- Service URL Policy: Fixed graceful rollout issues with ServiceURLGenerationPolicyOrdinalPrefix
- Better Error Handling: Improved error handling during rollout processes
Compaction and Storage
- Compaction Replicas: Updated compaction replicas configuration
- Memory Management: Improved memory management for compaction services
- Storage Optimization: Better storage configuration handling
Monitoring and Observability
- UniLink Metrics: Exposed metrics port (9999) for UniLink monitoring
- Prometheus Integration: Built-in Prometheus annotations for metrics collection
- Cert Check Option: Added certificate check option to detector
- Error Logging: Enhanced error logging for OIDC issuers reconciliation
🐛 Bug Fixes
Security and CVE Fixes
- Golang Image Upgrade: Upgraded Golang image to fix CVE vulnerabilities
- CVE Fixes: Multiple CVE fixes across dependencies
- Security Hardening: Enhanced security across all components
UniLink and Schema Registry Fixes
- Schema Registry Endpoint: Fixed UniLink Schema Registry endpoint configuration
- Subject Mapping: Fixed subject mapping configurations
- Endpoint Resolution: Improved endpoint resolution for schema registries
Gateway and Networking Fixes
- PFSQL Istio Hostname: Fixed PFSQL cluster Istio hostname with gateways defined
- Auth Policy Ports: Fixed authentication policy ports when multiple gateways configured
- Topology Gateway: Fixed topology-aware gateway selector configuration
Configuration and Deployment Fixes
- Manifest Fixes: Fixed manifests for master branch
- Client Configuration: Fixed Pulsar client.conf in toolset
- Requeue Logic: Added requeue for conflict errors
- Geo-Replication: Updated geo-replication metadata cleanup logic and version checks
Function and Worker Fixes
- Function Download: Fixed function download configuration
- Trusted Mode: Fixed trusted mode configuration for functions
- Worker Configuration: Improved function worker configuration
🚀 Performance Optimizations
Load Management
- Shadow Namespace: Optimized load management for Kafka-on-Pulsar workloads
- Bundle Configuration: Optimized namespace bundle settings (64 bundles)
- Memory Allocation: Improved memory allocation for managed ledger operations
- Load Balancing: Disabled unnecessary shedding for shadow namespace workloads
Data Migration Performance
- UniLink Optimization: Optimized UniLink data migration performance
- Schema Processing: Improved schema registry processing
- Topic Mapping: Enhanced topic mapping efficiency
Resource Management
- Memory Compute: Fixed memory compute issues
- Resource Allocation: Better resource allocation across components
- Compaction Scaling: Improved compaction replica scaling
🔄 Dependencies and Infrastructure
Kubernetes Upgrade
- K8s 1.30: Upgraded dependency to Kubernetes 1.30
- API Compatibility: Maintained API compatibility with newer Kubernetes versions
Security Updates
- Golang Upgrade: Upgraded Golang image for security fixes
- CVE Patches: Applied multiple CVE patches
- Dependency Updates: Updated various dependencies for security
Architecture Simplification
- RBAC Proxy Removal: Removed kube-rbac-proxy dependency
- Simplified Networking: Simplified networking configuration
- Port Management: Better port management and exclusion
📋 Configuration Changes
New Configuration Options
- UniLink Configuration: New UniLink and UniLinkSchema CRD configurations
- Shadow Namespace: New shadow namespace load manager configuration
- Function Controls: New function download and trusted mode controls
- Gateway Selectors: Additional selector configuration for topology-aware gateway
Enhanced Configuration
- S3 Table REST API: New S3 table REST API catalog configuration
- Subject Mappings: Flexible subject mapping configurations
- Schema Registry: Enhanced schema registry integration
- Memory Management: Improved memory management configurations
⚠️ Breaking Changes
- RBAC Proxy Removal: kube-rbac-proxy has been removed, which may affect existing RBAC configurations
- Port Management: Port 8443 is now excluded from Istio sidecar
- Load Manager Types: New shadow namespace load manager type available
- Function Configuration: New function download and trusted mode controls
📚 Documentation and Examples
- New UniLink data migration examples
- Shadow namespace load manager configuration guides
- Enhanced gateway configuration documentation
- Function management configuration examples
- S3 table REST API integration guides