Skip to main content

​
CLAUDE.md - StreamNative Client Documentation

This file provides guidance to Claude Code (claude.ai/code) when working with StreamNative client SDK documentation.

​
Purpose

The clients/ directory contains documentation for client libraries and SDKs that applications use to interact with StreamNative Cloud and Apache Pulsar, supporting both Pulsar native and Kafka protocols.

​
Directory Structure

  • kafka-clients/: Documentation for Kafka client libraries
    • Language-specific guides (Java, Go, Python, Node.js, C/C++, Spring Boot)
    • Kafka protocol compatibility and configuration
  • pulsar-clients/: Documentation for Pulsar native client libraries
    • Language-specific guides
    • Pulsar-specific features and patterns

​
Source Code Repository Mappings

​
Kafka Protocol Support

  • ksn (source_code_refs/ksn/): Kafka on StreamNative (KoP/KSN)
    • Implements Kafka protocol compatibility layer
    • Enables Kafka clients to work with Pulsar backends
    • Key directories:
      • kafka-impl/: Core Kafka protocol implementation
      • kafka-client-api/: Client API compatibility layer
      • oauth-client/: OAuth authentication for Kafka clients
      • docs/: KoP architecture and configuration guides
    • Important docs: docs/kop.md, docs/security.md, docs/schema.md
  • cloud-api-server: Provides authentication endpoints for clients
  • unified-rbac: Implements authorization for client connections
  • unilink: Enables cross-protocol client interoperability

​
Documentation Patterns

​
Client Documentation Structure

Each client language typically includes:
  1. Overview: Introduction and capabilities
  2. Installation: Package/dependency setup
  3. Connection: Authentication and connection configuration
  4. Producer: Sending messages with examples
  5. Consumer: Receiving messages with examples
  6. Advanced Features: Transactions, schemas, interceptors
  7. Best Practices: Performance tuning, error handling
  8. Troubleshooting: Common issues and solutions

​
Key Topics to Cover

​
Kafka Clients

  • OAuth2 authentication configuration
  • Schema Registry integration
  • Exactly-once semantics
  • Consumer group management
  • Partition assignment strategies
  • Compression and serialization
  • Interceptors and monitoring
  • Migration from Apache Kafka

​
Pulsar Clients

  • Token-based authentication
  • Schema evolution
  • Multi-topic subscriptions
  • Message acknowledgment patterns
  • Reader vs Consumer API
  • Delayed message delivery
  • Dead letter queues
  • Geo-replication awareness

​
SNIP References

Check source_code_refs/snip/proposals/ for design documents related to:
  • Kafka client compatibility (SNIP-101, SNIP-102, SNIP-110, SNIP-112, SNIP-122)
  • OAuth/OIDC authentication (SNIP-129, SNIP-136)
  • Schema registry improvements (SNIP-143)
  • Multi-protocol support (SNIP-147)

​
Common Tasks

​
Adding Client Examples

  1. Follow language-specific idioms and best practices
  2. Include complete, runnable examples
  3. Show both simple and advanced use cases
  4. Include error handling and retry logic
  5. Test examples against actual StreamNative clusters

​
Documenting Authentication

  1. Token authentication for Pulsar clients
  2. OAuth2 for Kafka clients (reference ksn/oauth-client/)
  3. Service account setup and API key generation
  4. TLS/mTLS configuration
  5. Integration with cloud provider identity systems

​
Performance Guidelines

  1. Connection pooling strategies
  2. Batch configuration for producers
  3. Parallelism for consumers
  4. Memory and buffer tuning
  5. Metrics collection and monitoring

​
Migration Guides

  1. Apache Kafka to StreamNative (Kafka protocol)
  2. Open-source Pulsar to StreamNative
  3. Version upgrade paths
  4. Feature compatibility matrices

​
Important Considerations

​
Multi-Protocol Support

  • Same data can be accessed via both Pulsar and Kafka APIs
  • Schema compatibility between protocols
  • Performance implications of protocol choice
  • Feature parity and limitations

​
Cloud vs Self-Hosted

  • Connection string differences
  • Authentication method variations
  • Available features by deployment type
  • Network and security configurations

​
Client Library Versions

  • Compatibility with server versions
  • Feature availability by version
  • Deprecated features and migration paths
  • Security patches and updates

​
Cross-References

  • Cloud docs for connection endpoints and authentication
  • Connect docs for connector development with clients
  • Tools docs for CLI client examples
  • API references for REST endpoints

​
Note on Source Code

The actual client SDK source code is maintained in separate repositories (not in source_code_refs/). This documentation references:
  • Protocol implementation (ksn)
  • Server-side support (cloud-api-server)
  • Authentication/authorization (unified-rbac)
For client SDK source code, refer to:
  • Apache Pulsar client repositories
  • StreamNative client SDK repositories
  • Kafka client libraries (when using KoP/KSN)

​
External Documentation References

  • Apache Pulsar Documentation: https://pulsar.apache.org/
    • Official Pulsar client documentation and API references
    • Architecture and concepts documentation
    • Best practices and deployment guides
  • Apache Kafka Documentation: https://kafka.apache.org/
    • Official Kafka client documentation
    • Protocol specifications
    • Configuration references for Kafka clients using KoP/KSN