Skip to main content
StreamNative Private Cloud is an enterprise product which brings specific controllers for Kubernetes by providing specific Custom Resource Definitions (CRDs) that extend the basic Kubernetes orchestration capabilities to support the setup and management of StreamNative components.

​
What’s Changed

​
🎉 Major New Features

​
Advanced Data Lake and Lakehouse Support (Brand New Feature)

  • Unity Catalog Integration: Full support for Databricks Unity Catalog with OAuth2 authentication
  • Snowflake Open Catalog: Native support for Snowflake open catalog integration
  • S3 Table Support: Support for external/managed S3 tables with proper secret management
  • Multi-Catalog Support: Support for multiple catalogs including Unity, Tabular, Polaris, and Open Catalog
  • Lakehouse Types: Support for Delta, Iceberg, and combined Delta+Iceberg lakehouse types
  • Table Modes: Support for both MANAGED and EXTERNAL table modes
  • Catalog Configuration: Flexible catalog configuration with URI and secret management

​
Enhanced Compaction Scheduler (Major Enhancement)

  • Azure DFS Support: Added support for Azure Data Lake Storage Gen2 (AzureDfs) backend storage
  • Pod Labels Support: Ability to specify custom pod labels for compaction scheduler
  • Resource Scaling: Compaction scheduler now scales to half of broker replicas for optimal performance
  • S3 Secret Management: Enhanced AWS S3 client secret mounting for compaction service
  • Region Unification: Unified region field configuration for Ursa and compaction services
  • NPE Fixes: Fixed CompactionScheduler null pointer exceptions

​
Ursa Cloud Storage Enhancements (Major Improvement)

  • Generic Bucket Configuration: Support for generic bucket configurations beyond S3
  • Region Field Unification: Unified region field handling across Ursa and compaction
  • Secret Path Management: Improved secret mount path handling when Ursa is enabled
  • AWS Region Override: Support for overwriting AWS region for Polaris catalog
  • Lakehouse Integration: Enabled lakehouse type support on Ursa and broker services

​
Advanced Reconciliation Controls (New Feature)

  • Coordinator Pause: Support for pausing Pulsar coordinator reconciliation with annotations
  • Oxia Shard Sorting: Fixed Oxia shard reconciliation with proper sorting
  • Namespace Reconciliation: Improved Oxia namespace reconciliation logic
  • Cleanup Logic: Enhanced replication cluster cleanup logic for new Pulsar versions

​
🔧 Feature Improvements

​
Graceful Rollout Enhancements

  • Label Fixes: Fixed graceful rollout label handling
  • Function Owner: Fixed function owner assignment during graceful rollout
  • Servlet Support: Enhanced servlet support for graceful rollout operations

​
Toolset and Resource Management

  • Memory Limits: Increased default memory limits for toolsets
  • Pod Resources: Added pod resource configuration for toolset
  • Auto Release: Support for automatically getting latest releases

​
Configuration Management

  • Backend Storage Type: Corrected compaction backend storage type configuration names
  • Log4j Sync: Synchronized Pulsar log4j configuration
  • Rest Message API: Proper handling when REST message API is explicitly disabled
  • Secret Path Updates: Updated secret paths and added S3 region field support

​
Istio and Networking

  • Zone Affinity: Upgraded to topology.kubernetes.io/zone label for better affinity
  • HTTPS Protocol: Fixed Istio HTTPS protocol handling
  • Multi-Address Support: Support for multi-address configuration for global ZooKeeper
  • Resource Conversion: Added Istio v1beta resource conversion tests

​
🐛 Bug Fixes

​
Security and CVE Fixes

  • Go Version Upgrade: Upgraded Go version to fix CVE vulnerabilities
  • Crypto Upgrades: Upgraded crypto libraries to fix CVE-45337 and other security issues
  • Golang Module: Upgraded golang.org/x/net module to fix CVE vulnerabilities

​
Compaction and Storage Fixes

  • S3 Secret Mounting: Fixed S3 secret mounting for S3 tables
  • Compaction NPE: Fixed CompactionScheduler null pointer exceptions
  • Storage Path: Fixed mount path for custom storage configurations
  • Backend Storage: Corrected compaction backend storage type configuration names

​
Reconciliation and Controller Fixes

  • Oxia Shards: Fixed Oxia shard reconciliation with proper sorting
  • Graceful Rollout: Fixed graceful rollout label handling
  • Function Owner: Fixed function owner assignment during graceful rollout
  • Replication Cleanup: Disabled reconcile cleanup for replication clusters in new Pulsar versions

​
Configuration and Deployment Fixes

  • Rest Message API: Fixed handling when REST message API is explicitly disabled
  • Log4j Config: Synchronized Pulsar log4j configuration
  • Secret Management: Improved secret path and region field handling
  • Init Container: Removed init container for autorecovery

​
🚀 Performance Optimizations

​
Compaction Performance

  • Resource Scaling: Compaction scheduler now scales to half of broker replicas
  • S3 Integration: Enhanced AWS S3 client secret mounting for better performance
  • Region Optimization: Unified region field handling for better resource utilization

​
Reconciliation Performance

  • Shard Sorting: Fixed Oxia shard sorting for better reconciliation performance
  • Pause Controls: Added ability to pause coordinator reconciliation when needed
  • Cleanup Logic: Improved replication cluster cleanup logic

​
Resource Management

  • Memory Limits: Increased default memory limits for toolsets
  • Pod Resources: Better pod resource allocation
  • Auto Scaling: Improved auto-scaling capabilities

​
🔄 Dependencies and Infrastructure

​
Security Updates

  • Go Version: Upgraded Go version to fix CVE vulnerabilities
  • Crypto Libraries: Upgraded crypto libraries for security fixes
  • Golang Modules: Updated golang.org/x/net module

​
CI/CD Improvements

  • JQ Installation: Fixed JQ installation in CI workflows
  • Disk Cleanup: Improved CI disk cleanup processes
  • Test Coverage: Added Istio v1beta resource conversion tests

​
External Dependencies

  • Function Mesh: Updated function mesh configurations
  • Kafka Connect: Fixed image capabilities for Kafka Connect
  • PFSQL: Updated PFSQL integration

​
📋 Configuration Changes

​
New Configuration Options

  • Lakehouse Type: New lakehouse type configuration for Delta, Iceberg support
  • Catalog Configuration: New catalog configuration for Unity, Snowflake, S3 tables
  • Table Modes: New table mode configuration (MANAGED/EXTERNAL)
  • Region Fields: Unified region field configuration
  • Pod Labels: Compaction scheduler pod label configuration

​
Enhanced Configuration

  • Ursa Generic Bucket: Generic bucket configuration support
  • S3 Region: Enhanced S3 region field support
  • Secret Paths: Updated secret path configurations
  • Backend Storage: Corrected backend storage type names

​
⚠️ Breaking Changes

  • Compaction Backend Storage: Backend storage type configuration names have been corrected
  • Secret Mounting: S3 secret mounting behavior changed for S3 tables
  • Region Fields: Region field handling has been unified across Ursa and compaction
  • Graceful Rollout Labels: Graceful rollout label handling has been updated

​
📚 Documentation and Examples

  • New Unity Catalog configuration examples
  • Snowflake Open Catalog integration guides
  • S3 table configuration documentation
  • Lakehouse type configuration examples
  • Multi-catalog setup guides