tests/config/readonly.xml → tests/ctest/config/readonly.xml

These configs are used in Docker images for tests.

FYI the tests for external models are not automated. It's time to convert them to integration tests.

tests/config/readonly.xml → tests/ctest/config/readonly.xml

These configs are used in Docker images for tests.

I see only one way how this configs are used in docker images.
All of them are included in clickhouse-test Debian package. Thus this configs are not a part of Docker image and test, but change with ClickHouse package itself.
dpkg -i package_folder/clickhouse-test_*.deb

And proller@ showed me before that this files are used in clickhoue-test package as well, thus the PR has to be fixed.

There is no documentation about clickhouse-test deb. It is our tests implementation detail and I believe we are free to change this package in a way we want to.

I consider more straightforward way to depend on configuration in tests and include them explicitly in tests.
Currently we have this lists of commands to setup correct environment for each type of tests:
https://gist.github.com/qoega/74cbb585cb3a2c5c7e67b0a3cf9ed2a8

Currently if we just need to add new config for test we have to

1. Commit new config to master
2. Commit new ln -s ... rows in scripts that are used in docker also in master
3. Wait until docker container is built and released
4. Add config in branch/PR
5. Build complete clickhouse package list to get clickhouse-test package and run test with it.

And environment that is used to run stateless/stateful tests manually differs from CI.

I believe we should make something as simple as queries directory that is used in tests.

• It can have several subdirectories with needed environments(for example default, stress etc.)
• We can use symlinks from non default env to default one. It is trivial to find what is changed in non-default env.
• Most tests should run in default env.
• No more way to lose new config in docker test definition(for example state{ful,less}_with_coverage)
• No need to build clickhouse-test package and clickhouse itself to check configuration change
• We can leave directory with configs in clickhouse-test for compatibility, but is would have default env as its source.
• Integrational tests that change configs use one of presets and explicitly have their own configuration files that are used only in this test.

If we do so, our tests would be more portable, and we would be able to re-run test with older clickhouse binaries.

TLDR:

• clickhouse-test should remain only as a package that allows to run all tests without additional dependencies
• All docker images for tests should use ClickHouse repo with source code as source of truth for queries, configs etc.
• Tests and clickhouse-test package use the same source of configuration.

Ok, let's make all builds successfull and merge.

PS. I don't care about clickhouse-test package at all.

Let's merge with master.

Something wrong with build scripts:

File "../clickhouse-server_0.0.0.3562_all.deb" not found.
./utils/release/release_lib.sh: line 194: cd: clickhouse-server-0.0.0.3562: No such file or directory

Please open a new PR.