- Process Data Streams
- pfSQL (Alpha)
pfSQL (Alpha) Overview
Note
This feature is currently in alpha. If you want to try it out or have any questions, submit a ticket to the support team.
What is pfSQL?
Built on top of Pulsar functions, pfSQL is a lightweight SQL-like tool that simplifies real-time data processing on StreamNative Cloud.
Unlike standard SQL, pfSQL queries streaming data, meaning that data is constantly flowing through them, either being read, or transformed and redirected. With pfSQL, you can set up a processing pipeline by writing queries with custom processing logic, where several operations like filtering, routing, and field projection can be chained and performed sequentially.
pfSQL allows you to query the following fields of a message with
- custom const values in string, number, or boolean format.
- message key (if any)
- message property
- message payload
What can pfSQL do?
Route traffic to different topics
Routing traffic to specific destinations for further analysis can be used for analytics, reporting, and lightweight application development. With pfSQL, you can route the query results to one or multiple Pulsar topics based on keys, properties, or fields.
Filter traffic to save bandwidth
Select a subset of messages that meet specific criteria. For example, filtering based on certain key, property, and field values or exact field matches.
Project payload fields for privacy purposes
Build a new message by transforming the format of a message, for example, extracting specific fields, and assembling and renaming them in the output result for downstream consumption.