Skip to content

Fix 00956_sensitive_data_masking flackiness#13867

Merged
alexey-milovidov merged 1 commit intoClickHouse:masterfrom
azat:00956_sensitive_data_masking-fix
Aug 19, 2020
Merged

Fix 00956_sensitive_data_masking flackiness#13867
alexey-milovidov merged 1 commit intoClickHouse:masterfrom
azat:00956_sensitive_data_masking-fix

Conversation

@azat
Copy link
Copy Markdown
Member

@azat azat commented Aug 18, 2020

Changelog category (leave one):

  • Not for changelog (changelog entry is not required)

00956_sensitive_data_masking is still flacky even after #13748 1

The problem is that it uses the following pattern:

clickhouse-client -q ... & # run some query in background
clickhouse-client -q 'show processlist' > log
grep background-query log

But there is no guarantee that the query in background will be executed before show processlist:

2020.08.18 02:52:47.916386 [ 26788 ] {98c36d38-f710-4dfb-af8f-61906abc163c} <Debug> executeQuery: (from [::1]:51650) SHOW PROCESSLIST
...
2020.08.18 02:52:47.926854 [ 26756 ] {086c64fa-713b-4f8c-b702-23bfea10a49c} <Debug> executeQuery: (from [::1]:51652) select count() from system.numbers where ignore('find_me_[hidden]')=0 and ignore('fwerkh_that_magic_string_make_me_unique') = 0 FORMAT Null

Fix the test by waiting until the query in backgroud will start, and use
limited numbers + sleepEachRow over system.numbers to reduce CPU usage.

00956_sensitive_data_masking is still flacky even after ClickHouse#13748 [1].

    [1]: https://clickhouse-test-reports.s3.yandex.net/10373/348ef1256ea8fb8f61109c33bbdd28daf46bdc8e/functional_stateless_tests_(debug).html#fail1

The problem is that it uses the following pattern:

    clickhouse-client -q ... & # run some query in background
    clickhouse-client -q 'show processlist' > log
    grep background-query log

But there is no guarantee that the query in background will be executed before `show processlist`:

    2020.08.18 02:52:47.916386 [ 26788 ] {98c36d38-f710-4dfb-af8f-61906abc163c} <Debug> executeQuery: (from [::1]:51650) SHOW PROCESSLIST
    ...
    2020.08.18 02:52:47.926854 [ 26756 ] {086c64fa-713b-4f8c-b702-23bfea10a49c} <Debug> executeQuery: (from [::1]:51652) select count() from system.numbers where ignore('find_me_[hidden]')=0 and ignore('fwerkh_that_magic_string_make_me_unique') = 0 FORMAT Null

Fix the test by waiting until the query in backgroud will start, and use
limited numbers + sleepEachRow over system.numbers to reduce CPU usage.
@robot-clickhouse robot-clickhouse added the pr-not-for-changelog This PR should not be mentioned in the changelog label Aug 18, 2020
@alexey-milovidov alexey-milovidov merged commit 1c40e3c into ClickHouse:master Aug 19, 2020
@azat azat deleted the 00956_sensitive_data_masking-fix branch August 19, 2020 19:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-not-for-changelog This PR should not be mentioned in the changelog

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants