Skip to content

Fix Digest does not match (from Replicated database) on CI#78741

Merged
azat merged 7 commits intoClickHouse:masterfrom
azat:rdb-digest-fix-2
Apr 8, 2025
Merged

Fix Digest does not match (from Replicated database) on CI#78741
azat merged 7 commits intoClickHouse:masterfrom
azat:rdb-digest-fix-2

Conversation

@azat
Copy link
Copy Markdown
Member

@azat azat commented Apr 6, 2025

Changelog category (leave one):

  • Not for changelog (changelog entry is not required)

The probelm was the 03001_matview_columns_after_modify_query test, and
only in stress tests, since there we don't set
database_replicated_always_detach_permanently=1, and so DETACH that is
done before playing with metadata on disk fails, and after this this
LOGICAL_ERROR will happen:

$ ../tests/clickhouse-test --client-option distributed_ddl_output_mode=none --replicated-database 03001_matview_columns_after_modify_query
...
Code: 49. DB::Exception: Received from localhost:9000. DB::Exception: Digest does not match. (LOGICAL_ERROR)
(query: ALTER TABLE mv MODIFY QUERY SELECT Timestamp as timestamp, c1 || c2 as c12 FROM src)

So, let's simply disable it.

Note it does not make a lot of sense to fix this DETACH error (with i.e.
PERMANTENTLY) since we need to modify the data in ZooKeeper additionally
for Replicated, plus it still can be racy if other tests will have "mv"
table.

Fixes: #64936

@azat azat marked this pull request as draft April 6, 2025 19:30
@clickhouse-gh
Copy link
Copy Markdown
Contributor

clickhouse-gh bot commented Apr 6, 2025

Workflow [PR], commit [0962e07]

@clickhouse-gh clickhouse-gh bot added the pr-not-for-changelog This PR should not be mentioned in the changelog label Apr 6, 2025
@azat azat changed the title RDB: Fix "Digest does not match" LOGICAL_ERROR (WIP) Improvements for debug digest check in Replicated database Apr 7, 2025
@azat azat marked this pull request as ready for review April 7, 2025 15:57
@azat azat enabled auto-merge April 7, 2025 15:57
@azat
Copy link
Copy Markdown
Member Author

azat commented Apr 7, 2025

@azat azat disabled auto-merge April 7, 2025 20:57
@azat azat marked this pull request as draft April 7, 2025 20:58
@azat azat force-pushed the rdb-digest-fix-2 branch from d4373ac to 3a09799 Compare April 8, 2025 10:19
@azat azat changed the title Improvements for debug digest check in Replicated database Fix Digest does not match (from Replicated database) on CI Apr 8, 2025
azat added 2 commits April 8, 2025 12:51
The probelm was the 03001_matview_columns_after_modify_query test, and
only in stress tests, since there we don't set
database_replicated_always_detach_permanently=1, and so DETACH that is
done before playing with metadata on disk fails, and after this this
LOGICAL_ERROR will happen:

    $ ../tests/clickhouse-test --client-option distributed_ddl_output_mode=none --replicated-database 03001_matview_columns_after_modify_query
    ...
    Code: 49. DB::Exception: Received from localhost:9000. DB::Exception: Digest does not match. (LOGICAL_ERROR)
    (query: ALTER TABLE mv MODIFY QUERY SELECT Timestamp as timestamp, c1 || c2 as c12 FROM src)

So, let's simply disable it.

Note it does not make a lot of sense to fix this DETACH error (with i.e.
PERMANTENTLY) since we need to modify the data in ZooKeeper additionally
for Replicated, plus it still can be racy if other tests will have "mv"
table.
@azat azat force-pushed the rdb-digest-fix-2 branch from 3a09799 to eb0c632 Compare April 8, 2025 10:52
@azat azat marked this pull request as ready for review April 8, 2025 10:52
azat added 4 commits April 8, 2025 13:35
We already have
tryCompareLocalAndZooKeeperTablesAndDumpDiffForDebugOnly() that compares
local vs zookeeper
Previously it prints only in-memory and in ZooKeeper, but all of our
problems on CI was when on-disk differs, so let's add it as well

And also I simplified (I hope) code a little.
@azat azat enabled auto-merge April 8, 2025 15:47
@azat azat added this pull request to the merge queue Apr 8, 2025
Merged via the queue into ClickHouse:master with commit 302f7c3 Apr 8, 2025
119 checks passed
@azat azat deleted the rdb-digest-fix-2 branch April 8, 2025 16:11
@robot-clickhouse robot-clickhouse added the pr-synced-to-cloud The PR is synced to the cloud repo label Apr 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-not-for-changelog This PR should not be mentioned in the changelog pr-synced-to-cloud The PR is synced to the cloud repo

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Replicated database engine: Assert checkDigestValid

3 participants