add docs about insertion deduplication on retries#2394
Conversation
| That transformed data also has to be deduplicated on retries. Clickhouse deduplicates it in the same way it deduplicates data which is inserted into target table. | ||
| User could control that process with settings on the table under materialized view: `replicated_deduplication_window`, `replicated_deduplication_window_seconds` and `non_replicated_deduplication_window`. Also user could use profile setting `deduplicate_blocks_in_dependent_materialized_views`. | ||
| For the blocks inserted in tables under materialized views Clickhouse calulates `block_id` as hash from a string which contains concatenation of `block_id`'s for the the source table and other parts which helps to distiguish blocks afret materialised views transformation, like source view's id and the sequential number of that block. That makes deduplication for materialised views works correctly and distinguish the data by its original inserted data, no mater how it has been transormed on its way to the destination table under materialized view. | ||
|
|
There was a problem hiding this comment.
Also mention that update_insert_deduplication_token_in_dependent_materialized_views is deprecated after some version?
Co-authored-by: Kseniia Sumarokova <[email protected]>
Co-authored-by: Kseniia Sumarokova <[email protected]>
Co-authored-by: Kseniia Sumarokova <[email protected]>
Co-authored-by: Kseniia Sumarokova <[email protected]>
Co-authored-by: Kseniia Sumarokova <[email protected]>
Co-authored-by: Kseniia Sumarokova <[email protected]>
Co-authored-by: Kseniia Sumarokova <[email protected]>
Co-authored-by: Kseniia Sumarokova <[email protected]>
Co-authored-by: Kseniia Sumarokova <[email protected]>
Co-authored-by: Kseniia Sumarokova <[email protected]>
|
Did this docs go to the production? The feature has not been merged yet... |
|
Sorry about that, will revert asap. I'll just hide it from the sidebar and it won't be accessible. I thought you wanted me to edit it and then merge. |
|
Yes, I want that. I should have filled the description for that PR as well, sorry. |
Did you reverted it? |
|
Yes this was removed |
|
The code is committed, I resurrect this doc in a day or two. |
3 participants
No description provided.