1. Setup Pulsar Clusters

BYOC - Set up AWS DNS

This document provides options on DNS in StreamNative cloud, as well as instructions on preparing a custom domain for use in StreamNative Cloud BYOC.

Domain options

StreamNative Cloud BYOC has two options for domains, Custom Domains and StreamNative Managed Domains.

In a custom domain, the customer delegates a zone for use by StreamNative. All endpoints will be created under that domain. With a StreamNative managed domain, a domain under the StreamNative snio.cloud domain is used, with a subzone delegated to the customer cloud account. For both custom domains and StreamNative-provided domains, StreamNative deploys the Kubernetes services external-dns and cert-manager to managed DNS and TLS certificates.

The following diagram depicts both domain options.

image of BYOC AWS DNS

Tip

  • The above diagram defines the custom domain as sncloud.<custom_domain> as an example.
  • The domain name delegated to StreamNative is up to you.

Custom domains

A custom domain allows for StreamNative clusters to have DNS names and TLS certificates live under a domain you control.

StreamNative requires a route53 zone for this custom domain. In general, it is recommended to have a dedicated route53 zone to limit only records required by StreamNative Cloud to be modified.

The AWS policies used by StreamNative can then be limited to only interacting with this route53 zone.

StreamNative managed domains

When StreamNative manages a domain, the route53 zone is created and managed by StreamNative in your AWS account under the snio.cloud domain. The subdomain is based on your organization ID. For example, an organization ID of o-12345 would have a domain of o-12345.snio.cloud.

StreamNative then delegates to this subdomain.

Note

Due to limitations with AWS IAM, when using StreamNative-managed domains, the AWS Access Module must first be provisioned with the route53 zone argument set to a wildcard (*). After the zone is created and the ID is known, this can then be limited to the zone ID.

Create a zone for a custom domain

StreamNative requires a route53 zone. It is strongly recommended to use a dedicated route53 zone. This section describes how to delegate a subzone from an existing domain. The instructions here assume that you are using route53 as your DNS provider. For details about how to delegate a subzone, see your DNS provider docs.

Create a subzone via delegation

The most common solution is to create a subdomain and delegate it from the parent domain.

If your parent domain is on route53, see the route53 documentation.

If your parent domain is not on route53, the following document provides general instructions.

Register a new domain

If you would prefer to register a new domain through route53, see the AWS documentation.

Validate the zone

In all instances, after the zone is created and the records have been delegated, we want to validate the zone.

Command line tools, such as dig, allow for directly querying DNS records and ensuring that the NS records have propagated.

This example assumes that you have created a subzone sncloud.myco.com, and that had NS records from route53 of:

  • ns-1654.awsdns-14.co.uk.
  • ns-1513.awsdns-61.org.
  • ns-449.awsdns-56.com.
  • ns-755.awsdns-30.net.

After creating the NS record from the parent domain, the dig NS sncloud.myco.com should return a response like:

$ dig NS sncloud.myco.com

; <<>> DiG 9.18.1-1ubuntu1.3-Ubuntu <<>> NS sncloud.myco.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 41380
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 9

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;sncloud.myco.com.		IN	NS

;; ANSWER SECTION:
sncloud.myco.com.	172800	IN	NS	ns-449.awsdns-56.com.
sncloud.myco.com.	172800	IN	NS	ns-755.awsdns-30.net.
sncloud.myco.com.	172800	IN	NS	ns-1513.awsdns-61.org.
sncloud.myco.com.	172800	IN	NS	ns-1654.awsdns-14.co.uk.

;; Query time: 251 msec
;; SERVER: 127.0.0.53#53(127.0.0.53) (UDP)
;; WHEN: Fri Feb 10 15:47:36 MST 2023
;; MSG SIZE  rcvd: 367

You can see that the ANSWER SECTION has 4 records that match the 4 NS records above.

Another option is to use a tool like DNS NS Lookup Checker to lookup records. However, this may take longer to validate.

Provide the zone ID

After you complete the validation, provide the zone ID to StreamNative. The format of the ID should be similar to ZXXXXXXXXXXXXXXXXXXXXX (the letter Z followed by 21 digits or uppercase letters).

Previous
Set up AWS Access