Infrastructure Failures

• Build check fails sporadically without any diagnostics @alesapin. Example: Fix build of test under Mac OS X #12772 and fix scalar subquery hash conflicts #8367 (added retries to all possible places)
• Sporadic failure in build: Error response from daemon: Get https://registry-1.docker.io/v2/yandex/clickhouse-deb-builder/manifests/latest: Get https://auth.docker.io/token?scope=repository%3Ayandex%2Fclickhouse-deb-builder%3Apull&service=registry.docker.io: EOF 2020-07-12 21:59:05,774 Cannot pull image yandex/clickhouse-deb-builder:latest and https://clickhouse-builds.s3.yandex.net/13947/cda85a78280c56cfd90ab96f04b28b56df3933da/clickhouse_build_check/build_log_753607578_1598022305.txt (added retries to all possible places)
• Docker prepare failed: https://sandbox.yandex-team.ru/task/708147271/view @alesapin and Fix assertion failure in FunctionsLogical #13213 (comment) (retries added in CI)
• Split build smoke test: Add missed <atomic> include in ProxyListConfiguration #12134 (comment) Need to add retries for all docker run / docker pull, etc. (retries added in CI)
• Cannot clone submodules: https://clickhouse-test-reports.s3.yandex.net/0/9c829e3046cae0711ca822998d1a6b3e59b54edb/clone_repo/submodules_init.err.log Need to add retries. Also a typo: Connot.
• Connection reset by peer while downloading test data from s3: Added a test for #11135 #12411 Need to add retries.
• Yandex Synchronization Check sometimes show bogus results @exprmntr
• TSan stress test sometimes killed by OOM: https://sandbox.yandex-team.ru/task/670887196/hardware
• Sometimes the check finish with SUCCESS but did not publish the result on GitHub due to GitHub API requests limit (5000 per hour).

Test Failures

• 01396_inactive_replica_cleanup_nodes -- link 1, link 2
• TestFlows LDAP @vzakaznikov @traceon
• test_dictionaries_all_layouts_and_sources/test.py::test_simple_dictionaries[3]. Incorrect result with thread sanitizer. https://clickhouse-test-reports.s3.yandex.net/0/4a73fe7477d52d806d71d2b2d57a8a2ca6e679e9/integration_tests_(thread).html. @alesapin and https://clickhouse-test-reports.s3.yandex.net/9516/2cda3c962ceb1c420919e3a187b89ef746421e9c/integration_tests_(asan).html#fail1
• Logical error, can be reproduced with 00993_system_parts_race_condition_drop_zookeeper and https://clickhouse-test-reports.s3.yandex.net/0/90b5e5d2f9edbcd4f01bd596ec919a32368fd716/functional_stateless_tests_(debug).html#fail1 and https://clickhouse-test-reports.s3.yandex.net/12314/260bcb9d795e2ea5607fd6a50481b4c2698c0249/functional_stateless_tests_(debug).html#fail1
• test_zookeeper_config/test.py::test_secure_connection broken with tsan link thread sanitizer trace
• logical error in ReplicatedMergeTreeQueue, 01035_concurrent_move_partition_from_table_zookeeper: https://clickhouse-test-reports.s3.yandex.net/9813/da7fa82ee0746fe70906476cd5f474f6a600ebbc/functional_stateless_tests_(debug).html#fail1
• topK functional test: https://clickhouse-test-reports.s3.yandex.net/0/4bb3381bc997f29bfbb412e75290c8036c843660/functional_stateless_tests_(release).html#fail1 and https://clickhouse-test-reports.s3.yandex.net/12278/e1543c18414e566fe7cbaaa3342e061357dfbf51/functional_stateless_tests_(release,_databaseatomic)/test_run.txt.out.log
• No route to host -- test_mysql_protocol - https://clickhouse-test-reports.s3.yandex.net/11163/ec894e7cb3e0e3f9f7c880b00d44ee3b83e028ad/integration_tests_(asan).html#fail1
• live view tests https://clickhouse-test-reports.s3.yandex.net/9813/05eed28b775e0029e6ccbe4655411e34681c1b67/functional_stateless_tests_(release,_polymorphic_parts_enabled).html#fail1 and https://clickhouse-test-reports.s3.yandex.net/9515/f6b69949589b83df1d7db24c5ae9d305a9fceb3b/functional_stateless_tests_(address).html#fail1 and https://clickhouse-test-reports.s3.yandex.net/9515/f6b69949589b83df1d7db24c5ae9d305a9fceb3b/functional_stateless_tests_(memory).html#fail1 and https://clickhouse-test-reports.s3.yandex.net/12055/b5fbd62a6f3c8cc426c82b77702d19e936315059/functional_stateless_tests_(release,_databaseatomic).html#fail1
• 00652_replicated_mutations_zookeeper.stdout https://clickhouse-test-reports.s3.yandex.net/0/d2be6c036b3b2f4bec8de985a9c620f9ae2cd28d/functional_stateless_tests_(release).html
  and https://clickhouse-test-reports.s3.yandex.net/14363/efd59eca62465e4dff860a588683215a4acdb353/functional_stateless_tests_(ubsan).html#fail1
• test_distributed_respect_user_timeouts/test.py::test_reconnect[configs-default-node2-remote] +
  [configs-default-node1-remote] + [configs_secure-default-node2-remote]
  https://clickhouse-test-reports.s3.yandex.net/0/ff0262626ab08919909d967b2859ee107656e659/integration_tests_(thread).html https://clickhouse-test-reports.s3.yandex.net/0/1a3a19391c144c3aed96d29db8bb47f5b0b899d7/integration_tests_(thread).html
• ThreadPool.ExceptionFromSchedule https://clickhouse-test-reports.s3.yandex.net/11718/03f39fe1f434d7541ab7f2cd4e9149304fda7b19/unit_tests_relwithdebuginfo_gcc-9.html#fail1
• 01064_incremental_streaming_from_2_src_with_feedback https://clickhouse-test-reports.s3.yandex.net/0/561d4b5b0108b300746d9a03978d7dfa75d58a36/functional_stateless_tests_(release,_databaseatomic).html
• use-of-uninitialized-value DB::parseIPv6 https://clickhouse-test-reports.s3.yandex.net/0/e460d7c2bd58b379cc68a2a2e144305af36f9c02/stress_test_(memory).html
• 00738_lock_for_inner_table https://clickhouse-test-reports.s3.yandex.net/12045/987e64acfe789689b8710add5d04371b8f21b604/functional_stateless_tests_(debug).html#fail1
• 01318_long_failing_mutation_zookeeper https://clickhouse-test-reports.s3.yandex.net/0/6f2d9999be75efdab1d4753620d6943592e674d0/functional_stateless_tests_(unbundled).html (someone just stopped merges globally and doesn't start them back, probably it's fixed)
• 01280_ttl_where_group_by https://clickhouse-test-reports.s3.yandex.net/0/ad53ef2c14da180fd56efa6439307e4650052a3d/functional_stateless_tests_(release,_databaseatomic).html
• multiSearch, MSan: https://clickhouse-test-reports.s3.yandex.net/12292/eaa13d81769af71d6c620881e7064ff47285c9aa/stress_test_(memory).html#fail1 (something related to Hyperscan?)
• 00943_materialize_index https://clickhouse-test-reports.s3.yandex.net/11831/4b7d253c8ab5225862e3ac4d443198e4d93072b6/functional_stateless_tests_(release,_polymorphic_parts_enabled)/test_run.txt.out.log and https://clickhouse-test-reports.s3.yandex.net/11936/ae9aaa006d9b2ce9972eb37ddebdd5476aa3a9c4/functional_stateless_tests_(memory).html#fail1 and https://clickhouse-test-reports.s3.yandex.net/12067/a541683efa81cd07d2bf6c27cdb65c5a9cc6b82b/functional_stateless_tests_(unbundled).html#fail1
• shutdown_table_after_detach: https://clickhouse-test-reports.s3.yandex.net/11901/53997f23e6cd34ec97c0e03d23a26184fdc5a633/functional_stateless_tests_(memory)/test_run.txt.out.log and https://clickhouse-test-reports.s3.yandex.net/0/f8229e0bf7edabc808a61bc9075795de534a4361/functional_stateless_tests_(memory).html and https://clickhouse-test-reports.s3.yandex.net/12704/eaaa4543f410d4a31243ab5e8b8ad404394625a7/functional_stateless_tests_(address)/test_run.txt.out.log and https://clickhouse-test-reports.s3.yandex.net/13228/9f07298eae3df1d981b04a20e475f511ebcbcf5f/functional_stateless_tests_(thread).html#fail1 and https://clickhouse-test-reports.s3.yandex.net/13737/082dbea0393774ef96c044db762133a8d895c814/functional_stateless_tests_(memory)/test_run.txt.out.log
• 01133_max_result_rows: https://clickhouse-test-reports.s3.yandex.net/11908/5a7e921135d27de93cf25479fd46a0eb7527ba3b/functional_stateless_tests_(release,_polymorphic_parts_enabled).html#fail1
• 01193_metadata_loading https://clickhouse-test-reports.s3.yandex.net/12052/cf306c5be13e6cf128e656086f8e0319e84c4de3/functional_stateless_tests_(thread).html#fail1
• Integration test: test_distributed_ddl/test.py::test_rename[configs_secure] https://clickhouse-test-reports.s3.yandex.net/0/5d2013f2240f450e25323a482e3a02312dc92849/integration_tests_(thread).html#fail1
• Hyperscan and async unwinding: https://clickhouse-test-reports.s3.yandex.net/12440/e8a3f0abb3e3169f0ade88bc2edd076ea4a0cff1/functional_stateful_tests_(release).html#fail1
• 00933_test_fix_extra_seek_on_compressed_cache https://clickhouse-test-reports.s3.yandex.net/12503/cd7ed94a085d1c810bff7a84abca1a1786e7e2a2/functional_stateless_tests_(release,_databaseatomic)/test_run.txt.out.log Maybe another read can happen during select query (system log)? and https://clickhouse-test-reports.s3.yandex.net/13209/7bd5505902f271eb9bd0a47291c6c9e617f44a31/functional_stateless_tests_(release,_databaseatomic).html#fail1 and https://clickhouse-test-reports.s3.yandex.net/13499/0e75eda5e6b4d50dd9ebfa537effcee9c106cb28/functional_stateless_tests_(release,_databaseatomic).html#fail1
• 00955_test_final_mark suspicious flap with database atomic https://clickhouse-test-reports.s3.yandex.net/12294/e35c49faa2ad63466ada9f29ea08a9fa3a022ba8/functional_stateless_tests_(release,_databaseatomic).html#fail1
• 00944_clear_index_in_partition (database atomic) https://clickhouse-test-reports.s3.yandex.net/0/21d3a797944066cb0b520e372570717ab165f3c4/functional_stateless_tests_(release,_databaseatomic).html#fail1
• 01040_dictionary_invalidate_query_switchover_long https://clickhouse-test-reports.s3.yandex.net/0/166a7b469ff7de99cd1d995eba950a31c52055a3/functional_stateless_tests_(release).html
• 00979_live_view_watch_live_moving_avg https://clickhouse-test-reports.s3.yandex.net/0/96fc213aeb33bf56483a7f5d9bdbce35fc969c66/functional_stateless_tests_(address).html
• 01444_create_table_drop_database_race https://clickhouse-test-reports.s3.yandex.net/13887/25dd0fa4d67daaf0db8d0f16965a6bf1204b3666/functional_stateless_tests_(release).html#fail1 Fix: 30e0cd4
• MemorySanitizer: use-of-uninitialized-value MemorySanitizer: use-of-uninitialized-value #14006