Skip to content

Properly remove projection from part in case it was removed from table metadata.#39119

Merged
KochetovNicolai merged 6 commits intomasterfrom
removed-projection-and-old-part
Jul 12, 2022
Merged

Properly remove projection from part in case it was removed from table metadata.#39119
KochetovNicolai merged 6 commits intomasterfrom
removed-projection-and-old-part

Conversation

@KochetovNicolai
Copy link
Copy Markdown
Member

@KochetovNicolai KochetovNicolai commented Jul 12, 2022

Changelog category (leave one):

  • Not for changelog (changelog entry is not required)

Closes #38629

@robot-ch-test-poll robot-ch-test-poll added the pr-not-for-changelog This PR should not be mentioned in the changelog label Jul 12, 2022
@tavplubix
Copy link
Copy Markdown
Member

tavplubix commented Jul 12, 2022

Seems like we can remove this (#39009) suppression now

@tavplubix tavplubix self-assigned this Jul 12, 2022
amosbird
amosbird reviewed Jul 12, 2022
View reviewed changes
tests/queries/0_stateless/01701_clear_projection_and_part_remove.sql Outdated
alter table tp_1 clear projection pp;
-- Remove projection from table metadata
alter table tp_1 drop projection pp;
-- Now, we don't load projection pp for attached part, bit it is written on disk
Copy link
Copy Markdown
Collaborator

@amosbird amosbird Jul 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bit -> but

@KochetovNicolai
Copy link
Copy Markdown
Member Author

KochetovNicolai commented Jul 12, 2022

AST fuzzer was likely killed by OOM killer

2022.07.12 15:36:13.198302 [ 169 ] {850ce5d5-bcbe-4e42-bc91-b2f7a1d1bba7} <Debug> executeQuery: (from [::ffff:127.0.0.1]:59726) SELECT '6553.5', arrayJoin(arrayMap(i -> (i + 10), range(1024))) AS index FROM numbers(65537) GROUP BY ['6553.5'], number ORDER BY arrayMap(i -> (i + -1), range(100)) ASC, arrayJoin(arrayMap(i -> (i + 0), range(2))) ASC, groupUniqArrayForEachStateArray([[NULL, NULL, NULL, NULL, NULL, NULL], [], []]) ASC NULLS FIRST, number ASC NULLS FIRST (stage: Complete)
2022.07.12 15:36:13.199295 [ 169 ] {850ce5d5-bcbe-4e42-bc91-b2f7a1d1bba7} <Trace> ContextAccess (default): Access granted: CREATE TEMPORARY TABLE ON *.*
2022.07.12 15:36:13.206555 [ 169 ] {850ce5d5-bcbe-4e42-bc91-b2f7a1d1bba7} <Trace> InterpreterSelectQuery: FetchColumns -> Complete
2022.07.12 15:36:13.212708 [ 400 ] {850ce5d5-bcbe-4e42-bc91-b2f7a1d1bba7} <Trace> AggregatingTransform: Aggregating
2022.07.12 15:36:13.212781 [ 400 ] {850ce5d5-bcbe-4e42-bc91-b2f7a1d1bba7} <Trace> Aggregator: An entry for key=13016873188886446729 found in cache: sum_of_sizes=65537, median_size=65537
2022.07.12 15:36:13.214533 [ 400 ] {850ce5d5-bcbe-4e42-bc91-b2f7a1d1bba7} <Trace> Aggregator: Aggregation method: key64
2022.07.12 15:36:13.262660 [ 400 ] {850ce5d5-bcbe-4e42-bc91-b2f7a1d1bba7} <Debug> AggregatingTransform: Aggregated. 65537 to 65537 rows (from 512.10 KiB) in 0.053511491 sec. (1224727.601 rows/sec., 9.35 MiB/sec.)
2022.07.12 15:36:13.262735 [ 400 ] {850ce5d5-bcbe-4e42-bc91-b2f7a1d1bba7} <Trace> Aggregator: Merging aggregated data
2022.07.12 15:36:14.586054 [ 400 ] {850ce5d5-bcbe-4e42-bc91-b2f7a1d1bba7} <Debug> MemoryTracker: Current memory usage (for query): 1.54 GiB.
2022.07.12 15:36:15.070136 [ 400 ] {850ce5d5-bcbe-4e42-bc91-b2f7a1d1bba7} <Debug> MemoryTracker: Current memory usage (for query): 2.04 GiB.
2022.07.12 15:36:15.941963 [ 400 ] {850ce5d5-bcbe-4e42-bc91-b2f7a1d1bba7} <Debug> MemoryTracker: Current memory usage (for query): 4.04 GiB.
2022.07.12 15:36:17.149047 [ 400 ] {850ce5d5-bcbe-4e42-bc91-b2f7a1d1bba7} <Debug> MemoryTracker: Current memory usage (for query): 36.04 GiB.
2022.07.12 15:36:17.578515 [ 400 ] {850ce5d5-bcbe-4e42-bc91-b2f7a1d1bba7} <Debug> MemoryTracker: Current memory usage (for query): 38.04 GiB.

2022-07-12 15:37:41	 Program terminated with signal SIGKILL, Killed.
2022-07-12 15:37:41	 The program no longer exists.

@KochetovNicolai
Copy link
Copy Markdown
Member Author

KochetovNicolai commented Jul 12, 2022

01643_replicated_merge_tree_fsync_smoke is flaky on master. Will disable it.

@KochetovNicolai
Copy link
Copy Markdown
Member Author

KochetovNicolai commented Jul 12, 2022

Integration tests are broken on master currently

@KochetovNicolai
Copy link
Copy Markdown
Member Author

KochetovNicolai commented Jul 12, 2022

Will not wait for tests, just fixed a typo in comment.

@KochetovNicolai KochetovNicolai merged commit 8efbe6d into master Jul 12, 2022
@KochetovNicolai KochetovNicolai deleted the removed-projection-and-old-part branch July 12, 2022 14:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@tavplubix tavplubix tavplubix approved these changes

+1 more reviewer

@amosbird amosbird amosbird left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

@tavplubix tavplubix

Labels

pr-not-for-changelog This PR should not be mentioned in the changelog

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

BC check: Cannot quickly remove directory ... by removing files

4 participants

@KochetovNicolai @tavplubix @amosbird @robot-ch-test-poll