Hi folks, we are excited to share our proposal for supporting a Generic Time Series Engine in Pinot: https://docs.google.com/document/d/1SBDDf71QZINYUjAbRSWguNMfbrWRGfdcF1JPi8SJZlM/edit

You can find a quick overview of the proposal below.

Context

At Uber, we have seen an increase in Observability use-cases for Pinot (for both alerting and monitoring). Pinot is commonly used for High-Cardinality Metrics use-cases at Uber, since our in-house system for Observability (M3), isn't optimized for them.

One of the feedbacks from the customers has been that SQL tends to get quite complex for time-series analysis, and it not only hampers productivity, but is also lacking certain features commonly available in Observability systems (esp. Gapfill).

Earlier this year, we internally built out native support in Pinot for M3QL, Uber's open-source query language. The results were quite promising: we were able to support all of M3QL's features natively in Pinot.

We hence kick-started this effort to bring these improvements to OSS Pinot.

Language Agnostic

Our proposal is such that one will be able to implement plugins that can re-use the Generic Time Series Engine to support any time-series query language (PromQL, M3QL, etc.). This will allow each stakeholder to develop their own language plugins.

For more please refer to the doc.

cc: @kishoreg