Test 03036_dynamic_read_subcolumns was failing in master. It's already fixed so I updated the branch with recent changes to avoid its failing in the CI

I'm quite interested in this implementation, is this still being driven in any way?

@Michicosun Really great work! Will you implement dataflow model https://www.oreilly.com/radar/the-world-beyond-batch-streaming-101, like time window, watermark, triggers, etc. ?

Hi @zhanglistar , if you are interested in watermark, out of order processing, etc, you may check our stream processing engine built with ClickHouse codebase https://github.com/timeplus-io/proton We are covered in another O'Reilly book https://www.oreilly.com/library/view/streaming-databases/9781098154820/ It's not exactly dataflow but can do many things similar to Apache Flink, but in C++ speed and low footprint. Happy to work with ClickHouse community to work on stream processing together.

Hi @zhanglistar , if you are interested in watermark, out of order processing, etc, you may check our stream processing engine built with ClickHouse codebase https://github.com/timeplus-io/proton We are covered in another O'Reilly book https://www.oreilly.com/library/view/streaming-databases/9781098154820/ It's not exactly dataflow but can do many things similar to Apache Flink, but in C++ speed and low footprint. Happy to work with ClickHouse community to work on stream processing together.

@jovezhong Thanks for your reply. I have seen proton, great job! And there is another streamming processing framework like arroyo, risingwave writen in Rust. I am studying these things, and also, we may impletement native flink task like https://developer.aliyun.com/article/1634363 in gluten project https://github.com/apache/incubator-gluten, and we already have a POC, but there still lots of work to do, like stateful functions.

Could this hypothetically interact with or build on top of #79417 and #79471? Those PRs combined seem to support a stable low-level notion of a cursor (_part_starting_offset + _part_offset) and a stable table snapshot to refer to across multiple (sub-)queries, though maybe this can be extended to work across requests for more rows from a stream.

Just wondering, because I have no real knowledge here but the above PRs seem to be useful for this feature.

Is this still planned for 2025?