Skip to content

Put ingester disable chunk trimming a feature flag #6300

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
yeya24 merged 6 commits into from
Nov 1, 2024
Merged

Put ingester disable chunk trimming a feature flag #6300

yeya24 merged 6 commits into from
Nov 1, 2024

Conversation

@yeya24
Copy link
Contributor

@yeya24 yeya24 commented Nov 1, 2024 •
edited
Loading

Signed-off-by: Ben Ye [email protected]<!-- Thanks for sending a pull request! Before submitting:

  1. Read our CONTRIBUTING.md guide
  2. Rebase your PR if it gets out of sync with master
    -->

What this PR does:

This changes #6270 to mark the behavior optional under a feature flag.

We noticed that query result got changed if disabling chunk trimming for the following test case, captured by fuzz test.

https://github.com/cortexproject/cortex/actions/runs/11621920479/job/32367371250?pr=6298

The failure seems correctly captured as timestamp function returns timestamp of the sample so if certain samples are not trimmed then the result changes.

I think the behavior is not intended but let's make disable chunk trimming a feature flag for now and users can enable it at their own risk

query_fuzz_test.go:188: case 296 results mismatch.
        range query: timestamp({__name__="test_series_3"} @ end() offset 3m21s)
        res1: {job="test"} =>
        1730421141.171 @[1730419941.189]
        1730421141.171 @[1730419971.189]
        1730421141.171 @[1730420001.189]
        1730421141.171 @[1730420031.189]
        1730421141.171 @[1730420061.189]
        1730421141.171 @[1730420091.189]
        1730421141.171 @[1730420121.189]
        1730421141.171 @[1730420151.189]
        1730421141.171 @[1730420181.189]
        1730421141.171 @[1730420211.189]
        1730421141.171 @[1730420241.189]
        1730421141.171 @[1730420271.189]
        1730421141.171 @[1730420301.189]
        1730421141.171 @[1730420331.189]
        1730421141.171 @[1730420361.189]
        1730421141.171 @[1730420391.189]
        1730421141.171 @[1730420421.189]
        1730421141.171 @[1730420451.189]
        1730421141.171 @[1730420481.189]
        1730421141.171 @[1730420511.189]
        1730421141.171 @[1730420[54](https://github.com/cortexproject/cortex/actions/runs/11621920479/job/32367371250?pr=6298#step:10:55)1.189]
        1730421141.171 @[1730420571.189]
        1730421141.171 @[1730420601.189]
        1730421141.171 @[1730420631.189]
        1730421141.171 @[1730420661.189]
        1730421141.171 @[1730420691.189]
        1730421141.171 @[1730420721.189]
        1730421141.171 @[1730420751.189]
        1730421141.171 @[1730420781.189]
        1730421141.171 @[1730420811.189]
        1730421141.171 @[1730420841.189]
        1730421141.171 @[1730420871.189]
        1730421141.171 @[1730420901.189]
        1730421141.171 @[1730420931.189]
        1730421141.171 @[1730420961.189]
        1730421141.171 @[1730420991.189]
        1730421141.171 @[1730421021.189]
        1730421141.171 @[1730421051.189]
        1730421141.171 @[1730421081.189]
        1730421141.171 @[1730421111.189]
        1730421141.171 @[1730421141.189]
        res2: {job="test"} =>
        1730420931.171 @[1730419941.189]
        1730420931.171 @[1730419971.189]
        1730420931.171 @[1730420001.189]
        1730420931.171 @[1730420031.189]
        1730420931.171 @[1730420061.189]
        1730420931.171 @[1730420091.189]
        1730420931.171 @[1730420121.189]
        1730420931.171 @[1730420151.189]
        1730420931.171 @[1730420181.189]
        1730420931.171 @[1730420211.189]
        1730420931.171 @[1730420241.189]
        1730420931.171 @[1730420271.189]
        1730420931.171 @[1730420301.189]
        1730420931.171 @[1730420331.189]
        1730420931.171 @[1730420361.189]
        1730420931.171 @[1730420391.189]
        1730420931.171 @[1730420421.189]
        1730420931.171 @[1730420451.189]
        1730420931.171 @[1730420481.189]
        1730420931.171 @[1730420511.189]
        1730420931.171 @[1730420541.189]
        1730420931.171 @[1730420[57](https://github.com/cortexproject/cortex/actions/runs/11621920479/job/32367371250?pr=6298#step:10:58)1.189]
        1730420931.171 @[1730420601.189]
        1730420931.171 @[1730420631.189]
        1730420931.171 @[1730420661.189]
        1730420931.171 @[1730420691.189]
        1730420931.171 @[1730420721.189]
        1730420931.171 @[1730420751.189]
        1730420931.171 @[1730420781.189]
        1730420931.171 @[1730420811.189]
        1730420931.171 @[1730420841.189]
        1730420931.171 @[1730420871.189]
        1730420931.171 @[1730420901.189]
        1730420931.171 @[1730420931.189]
        1730420931.171 @[17304209[61](https://github.com/cortexproject/cortex/actions/runs/11621920479/job/32367371250?pr=6298#step:10:62).189]
        1730420931.171 @[1730420991.189]
        1730420931.171 @[1730421021.189]
        1730420931.171 @[1730421051.189]
        1730420931.171 @[1730421081.189]
        1730420931.171 @[1730421111.189]
        1730420931.171 @[1730421141.189]
    query_fuzz_test.go:193: 
        	Error Trace:	/home/runner/work/cortex/cortex/integration/query_fuzz_test.go:193
        	Error:      	finished query fuzzing tests
        	Test:       	TestDisableChunkTrimmingFuzz
        	Messages:   	1 test cases failed

Which issue(s) this PR fixes:
Fixes #

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

@pull-request-size pull-request-size bot added size/M and removed size/S labels Nov 1, 2024
@yeya24
lint
5bbe234 
Signed-off-by: Ben Ye <[email protected]>
@yeya24 yeya24 force-pushed the disable-chunk-trimming branch from bce5678 to 5bbe234 Compare November 1, 2024 03:41
yeya24
yeya24 commented Nov 1, 2024
View reviewed changes
integration/query_fuzz_test.go
for {
expr = ps.WalkRangeQuery()
query = expr.Pretty(0)
if !strings.Contains(query, "timestamp") {
Copy link
Contributor Author

@yeya24 yeya24 Nov 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We on purposely ignore timestamp to reduce flakiness as it is a know imcompatible behavior

alanprot
alanprot approved these changes Nov 1, 2024
View reviewed changes
Copy link
Member

@alanprot alanprot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Nov 1, 2024
@yeya24 yeya24 merged commit 4063773 into cortexproject:master Nov 1, 2024
@yeya24 yeya24 deleted the disable-chunk-trimming branch November 1, 2024 16:36
CharlieTLe pushed a commit to CharlieTLe/cortex that referenced this pull request Dec 3, 2024
@yeya24 @CharlieTLe
Put ingester disable chunk trimming a feature flag (cortexproject#6300)
5876163
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@alanprot alanprot alanprot approved these changes

Assignees

No one assigned

Labels

component/ingester lgtm This PR has been approved by a maintainer size/M

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@yeya24 @alanprot