> ## Documentation Index
> Fetch the complete documentation index at: https://docs.streamnative.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Advanced Observability Integration

> [!NOTE]
>
> The advanced observability features, including the Local Metrics Endpoint and Remote Write Integration, are only available for [**BYOC Pro**](https://docs.streamnative.io/cloud/clusters/cluster-types.md#byoc-pro-clusters) clusters. Use the [Metrics API](https://docs.streamnative.io/cloud/log-and-monitor/cloud-metrics-api.md) for other clusters, including [Serverless](https://docs.streamnative.io/cloud/clusters/cluster-types.md#serverless-clusters), [Dedicated](https://docs.streamnative.io/cloud/clusters/cluster-types.md#dedicated-clusters), and [BYOC](https://docs.streamnative.io/cloud/clusters/cluster-types.md#byoc-clusters).
>
> If you want to access these advanced observability features, you can [contact StreamNative sales](https://www.streamnative.io/contact) to get a quote for BYOC Pro clusters.

In addition to the [Metrics API](https://docs.streamnative.io/cloud/log-and-monitor/cloud-metrics-api.md), StreamNative Cloud offers advanced observability features to help you gain deeper insights into your Pulsar clusters and applications. This document covers two key aspects of advanced observability:

1. Local Metrics Endpoint: A local Prometheus endpoint that is deployed with your BYOC Pro cluster. It provides access to all available metrics from your Pulsar clusters including both resource-related metrics and system-level metrics.

2. Remote Write Integration: Capabilities for seamlessly forwarding metrics to external monitoring and analytics platforms.

## Local Metrics Endpoint

The [Metrics API](https://docs.streamnative.io/cloud/log-and-monitor/cloud-metrics-api.md) can be used for monitoring and troubleshooting your business applications. You can get the metrics from your Pulsar clusters, including tenants/namespaces/topics, connectors, functions, and so on. These metrics focus on the Pulsar resources inside your cluster. Metrics API excludes system-level metrics, such as the broker, bookie, and Pulsar Functions worker metrics. StreamNative manages these system-level metrics for you to ensure the stability and reliability of your Pulsar clusters.

If you need to access these system-level metrics, you can use the Local Metrics Endpoint. The Local Metrics Endpoint provides access to all available metrics from your Pulsar clusters, including the system-level metrics.

> [!NOTE]
>
> The Local Metrics Endpoint is only available for **BYOC Pro** clusters.

### Enable the Local Metrics Endpoint

The Local Metrics Endpoint can be enabled on a per-cluster basis. Submit a **Change Request** in our [support portal](https://docs.streamnative.io/support/get-support.md) to enable the Local Metrics Endpoint for your cluster.

After the request is approved and the Local Metrics Endpoint is enabled, you will be provided with a Local Metrics Endpoint URL, in the form `https://metric-<id>.<your-domain>`.

### Scrape the Local Metrics Endpoint

The Local Metrics Endpoint serves the Prometheus federation API and authenticates with API keys, which differs from the [Metrics API](https://docs.streamnative.io/cloud/log-and-monitor/cloud-metrics-api.md). For the endpoint paths, the authentication header, collector configuration examples, and troubleshooting steps, see [Scrape Metrics from the Local Metrics Endpoint](https://docs.streamnative.io/cloud/log-and-monitor/local-metrics-endpoint.md).

## Metrics Remote Write Integration

For some reasons your observability stack can't pull from the [Metrics API](https://docs.streamnative.io/cloud/log-and-monitor/cloud-metrics-api.md), StreamNative Cloud supports pushing metrics data with Remote Write to Prometheus-compatible systems and Datadog platform.

The Metrics Remote Write is enabled on the [Cloud Environment](https://docs.streamnative.io/cloud/clusters/byoc/create-cloud-environment.md):

> [!NOTE]
>
> StreamNative Cloud does not yet support enabling the Metrics Remote Write on existing Cloud Environment through StreamNative Cloud Console.

![Metrics Remote Write](https://mintcdn.com/streamnative/DYzjgCK90kxVTpdd/media/o11y/metrics-remote-write.png?fit=max\&auto=format\&n=DYzjgCK90kxVTpdd\&q=85\&s=76455a206142040efd08c42c9996c175)

### Remote Write to Prometheus

![Prometheus Remote Write](https://mintcdn.com/streamnative/DYzjgCK90kxVTpdd/media/o11y/prometheus-remote-write.png?fit=max\&auto=format\&n=DYzjgCK90kxVTpdd\&q=85\&s=73f8c84b437280ac473cf056bf0c1a0a)

To enable the Prometheus Remote Write:

1. Enable the Metrics Remote Write switch.
2. Select the Prometheus icon.
3. Input the Prometheus endpoint address.
4. Select the Prometheus authentication type, for now supports Basic and Bearer token.

### Remote Write to Datadog

![Datadog Remote Write](https://mintcdn.com/streamnative/DYzjgCK90kxVTpdd/media/o11y/datadog-remote-write.png?fit=max\&auto=format\&n=DYzjgCK90kxVTpdd\&q=85\&s=6725e7fd07b86b0aa9c07b8f5e9c9ad2)

To enable the Datadog Remote Write:

1. Enable the Metrics Remote Write switch.
2. Select the Datadog icon.
3. Input the [Datadog Sites](https://docs.datadoghq.com/getting_started/site/).
4. Input the Datadog [API Key](https://docs.datadoghq.com/account_management/api-app-keys/#api-keys).

#### Recommended way to monitor Pulsar in Datadog

Metrics Remote Write is the recommended way to monitor your Pulsar clusters in Datadog. Metrics that arrive through Remote Write use the `pulsar.*` namespace in Datadog, for example `pulsar.pulsar_topics_count` and `pulsar.pulsar_msg_backlog`.

> [!NOTE]
>
> The `pulsar.` prefix is the Datadog namespace, and the rest of the name is the original metric name (which also begins with `pulsar_`). As a result, a metric such as `pulsar_msg_backlog` appears as `pulsar.pulsar_msg_backlog` in Datadog. This repetition is intentional.

To visualize these metrics, import the StreamNative-maintained dashboard from the [streamnative/pulsar-datadog](https://github.com/streamnative/pulsar-datadog) repository.

> [!WARNING]
>
> Don't enable Datadog's native StreamNative integration at the same time as Metrics Remote Write. The native integration pulls the same data from the [Metrics API](https://docs.streamnative.io/cloud/log-and-monitor/cloud-metrics-api.md) and ingests it under the separate `streamnative.pulsar_resource.*` namespace (for example, `streamnative.pulsar_resource.pulsar_topics_count`). Running both paths produces duplicate metrics and increases your Datadog ingestion costs. Choose one path, and prefer Metrics Remote Write.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.