Skip to content

Fix table function clusterAllReplicas return wrong _shard_num#21498

Merged
CurtizJ merged 2 commits intoClickHouse:masterfrom
ucasfl:fix-shard_num
Mar 29, 2021
Merged

Fix table function clusterAllReplicas return wrong _shard_num#21498
CurtizJ merged 2 commits intoClickHouse:masterfrom
ucasfl:fix-shard_num

Conversation

@ucasfl
Copy link
Copy Markdown
Collaborator

@ucasfl ucasfl commented Mar 6, 2021

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Changelog category (leave one):

  • Bug Fix

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Fix table function clusterAllReplicas returns wrong _shard_num. close #21481.

@robot-clickhouse robot-clickhouse added the pr-bugfix Pull request with bugfix, not backported by default label Mar 6, 2021
@ucasfl ucasfl force-pushed the fix-shard_num branch 2 times, most recently from 8beba28 to 37ff1f6 Compare March 7, 2021 13:06
@CurtizJ CurtizJ self-assigned this Mar 9, 2021
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks incorrect, because it will be incremented for all replicas and replicas from one shard will have different values.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's correct. This function only used for clusterAllReplicas which regard all replicas as independent shards.

Copy link
Copy Markdown
Collaborator Author

@ucasfl ucasfl Mar 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It create a new Cluster contains all replicas as shards.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's just add functional test that shows that everything is correct for multi-replica shards.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How can I add another cluster for test?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ucasfl You can edit tests/config.d/cluster.xml

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why I add cluster definition into server-test.xml or cluster.xml, the test still also show that cluster not found? @alexey-milovidov

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why I add cluster definition into server-test.xml or cluster.xml, the test still also show that cluster not found?

AFAIK server-test.xml is not used anywhere (except for arcadia maybe), if you will add your cluster to tests/config/config.d/clusters.xml it should work.

FWIW fast-test (and other checks) got this configs via tests/config/install.sh, it already includes clusters.xml, here is a snippet from fast-test log:

+ ln -sf /ClickHouse/tests/config/config.d/clusters.xml /fasttest-workspace/db-fasttest/config.d/

Previous variant (f6e83b5) - https://clickhouse-test-reports.s3.yandex.net/21498/f6e83b579cc2b9db37b0a6315ee5633eaa6cf86a/fast_test.html#fail1 was failed because you set port to 59000, and this port is used only for arcadia tests AFAIR.

So to summarize you should add everything like in f6e83b579cc2b9db37b0a6315ee5633eaa6cf86a, but with default port.

@CurtizJ
Copy link
Copy Markdown
Member

CurtizJ commented Mar 9, 2021

Also, maybe it will be useful to have virtual column _replica_num.

@ucasfl ucasfl force-pushed the fix-shard_num branch 5 times, most recently from f6e83b5 to 4beb362 Compare March 13, 2021 11:42
fix build on gcc-10

better

update test

fix

fix

fix

fix

fix

fix cluster config
Copy link
Copy Markdown
Member

@CurtizJ CurtizJ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok

@CurtizJ
Copy link
Copy Markdown
Member

CurtizJ commented Mar 29, 2021

Commit 9f3ca62 is shown in changes of this PR, but it's already in master. I think it's because of several force-pushes.

robot-clickhouse pushed a commit that referenced this pull request Mar 30, 2021
robot-clickhouse pushed a commit that referenced this pull request Mar 30, 2021
alexey-milovidov added a commit that referenced this pull request Mar 31, 2021
Backport #21498 to 21.3: Fix table function clusterAllReplicas return wrong _shard_num
CurtizJ added a commit that referenced this pull request Mar 31, 2021
Backport #21498 to 21.2: Fix table function clusterAllReplicas return wrong _shard_num
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-bugfix Pull request with bugfix, not backported by default

Projects

None yet

Development

Successfully merging this pull request may close these issues.

_shard_num and clusterAllReplicas()

7 participants