Feature Request: Add Support for NATS Jetstream in Proton/Timeplus

Use Case

Proton/Timeplus is recognized for its high-performance streaming SQL capabilities, largely powered by ClickHouse. To enhance its integration capabilities and messaging support, it would be beneficial to add support for NATS Jetstream. NATS Jetstream offers advanced features like persistence, at-least-once delivery, consumer offsets, and replayability, which are essential for reliable and scalable message streaming in complex, distributed systems. By integrating NATS Jetstream, Proton/Timeplus can be leveraged in more diverse environments, particularly where NATS is already an integral part of the messaging infrastructure.

Desired Solution

The ideal solution would be the implementation of a NATS Jetstream connector within Proton/Timeplus. This connector should be capable of both publishing to and subscribing from NATS Jetstream topics. Key features would include:

• Stream Ingestion: Ability to ingest data streams from NATS Jetstream, enabling real-time analytics on the streaming data.
• Fault Tolerance: Ensuring at-least-once delivery semantics to guarantee message processing in the face of network or system failures.
• Scalability: Support for NATS Jetstream scalability features, including partitioning and load balancing across multiple consumers.
• Ease of Configuration: Simple configuration setup for connecting Proton/Timeplus with a NATS Jetstream server, including stream and consumer configuration.
• Data Transformation and Enrichment: Capability to perform SQL-based transformations and enrichments on the ingested data from NATS Jetstream.

Alternatives Considered

Currently, integrating NATS Jetstream with Proton/Timeplus requires custom solutions or middleware, which can be complex and less efficient. Other messaging systems like Apache Kafka are supported, but they do not align with the infrastructure where NATS Jetstream is already in use.

Additional Context

The addition of NATS Jetstream support would significantly enhance Proton/Timeplus's appeal in environments where lightweight and high-performance messaging is crucial. It would also align Proton/Timeplus with modern cloud-native architectures that are increasingly adopting NATS Jetstream for messaging.