Skip to content

refactor streamsink and enhance StorageRandom.cpp‎#1057

Merged
yokofly merged 3 commits intodevelopfrom
bugfix/issue-1055-pause-mat-view
Dec 12, 2025
Merged

refactor streamsink and enhance StorageRandom.cpp‎#1057
yokofly merged 3 commits intodevelopfrom
bugfix/issue-1055-pause-mat-view

Conversation

@yokofly
Copy link
Copy Markdown
Collaborator

@yokofly yokofly commented Dec 11, 2025 •
edited
Loading

PR checklist:

  • Did you run ClangFormat ?
  • Did you separate headers to a different section in existing community code base ?
  • Did you surround proton: starts/ends for new code in existing community code base ?

Please write user-readable short description of the changes:

  1. unblock PAUSE random source MatView
src/Storages/Random/StorageRandom.cpp:624
 Chunk doCheckpoint(CheckpointContextPtr ckpt_ctx) override
    {
        IProcessor::checkpoint(ckpt_ctx);

        auto result = header_chunk.clone();
        result.setCheckpointContext(ckpt_ctx);
        return result;
    }
  1. correctly handle blocks
    see void StreamSinkBase::checkpoint(CheckpointContextPtr ckpt_ctx)

@yokofly yokofly linked an issue Dec 11, 2025 that may be closed by this pull request
failed to pause matview #1055
Closed
@yokofly yokofly marked this pull request as draft December 11, 2025 08:56
@yokofly yokofly marked this pull request as ready for review December 11, 2025 09:12
chatgpt-codex-connector[bot]
chatgpt-codex-connector bot reviewed Dec 11, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@yokofly yokofly marked this pull request as draft December 11, 2025 15:20
@yokofly
Add StreamSinkBase class for handling streaming data ingestion
2e6b439 
This commit introduces the StreamSinkBase class, which provides a foundation for managing data ingestion in a streaming context. It includes methods for consuming chunks of data, handling retries, and checkpointing. The StreamSink class is updated to inherit from StreamSinkBase, streamlining its implementation and enhancing code organization. This change lays the groundwork for improved data processing capabilities in the streaming architecture.
@yokofly
fix #1055 enhance random source
6ad7cf0
@yokofly
add test
367e86d
@yokofly yokofly force-pushed the bugfix/issue-1055-pause-mat-view branch from 05a3255 to 367e86d Compare December 12, 2025 03:20
@yokofly
Copy link
Copy Markdown
Collaborator Author

yokofly commented Dec 12, 2025

@codex review

@chatgpt-codex-connector
Copy link
Copy Markdown
Contributor

chatgpt-codex-connector bot commented Dec 12, 2025

To use Codex here, create a Codex account and connect to github.

@yokofly
Copy link
Copy Markdown
Collaborator Author

yokofly commented Dec 12, 2025

@codex review

chatgpt-codex-connector[bot]
chatgpt-codex-connector bot reviewed Dec 12, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@yokofly yokofly marked this pull request as ready for review December 12, 2025 03:29
@yokofly
Copy link
Copy Markdown
Collaborator Author

yokofly commented Dec 12, 2025

failed case

2025-12-12 04:45:47 99029_skip_poison_events_bug:                                           [ FAIL ] - result differs with reference: 
2025-12-12 04:45:47 --- /proton_src/tests/queries_ported/0_stateless/99029_skip_poison_events_bug.reference	2025-12-12 04:30:27.377104642 +0000
2025-12-12 04:45:47 +++ /proton_src/tests/queries_ported/0_stateless/99029_skip_poison_events_bug.stdout	2025-12-12 04:45:47.768386988 +0000
2025-12-12 04:45:47 @@ -4,3 +4,4 @@
2025-12-12 04:45:47  recover_policy=strict This test is expected to fail with a poison event
2025-12-12 04:45:47  1	1
2025-12-12 04:45:47  1	1
2025-12-12 04:45:47 +1	1
2025-12-12 04:45:47 
2025-12-12 04:45:47

@yokofly yokofly changed the title fix #1055 enhance random source refactor streamsink and enhance StorageRandom.cpp‎ Dec 12, 2025
@yokofly yokofly merged commit 258dce5 into develop Dec 12, 2025
27 of 28 checks passed
@yokofly yokofly deleted the bugfix/issue-1055-pause-mat-view branch December 12, 2025 05:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@chatgpt-codex-connector chatgpt-codex-connector[bot] chatgpt-codex-connector[bot] left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

failed to pause matview

1 participant

@yokofly