ClickHouse over web server with static files#25251
ClickHouse over web server with static files#25251kssenii merged 32 commits intoClickHouse:masterfrom
Conversation
robot-clickhouse
added
doc-alert
pr-feature
kssenii
force-pushed
the
disk-over-web-server
branch
2 times, most recently
from
2882341 to
0b79a60
Compare
kssenii
force-pushed
the
disk-over-web-server
branch
3 times, most recently
from
4fe79bc to
8f1fea1
Compare
kssenii
force-pushed
the
disk-over-web-server
branch
3 times, most recently
from
9d759c1 to
46d12ac
Compare
kssenii
force-pushed
the
disk-over-web-server
branch
from
46d12ac to
babb11a
Compare
kssenii
force-pushed
the
disk-over-web-server
branch
from
840f163 to
55f484e
Compare
kssenii
force-pushed
the
disk-over-web-server
branch
from
55f484e to
5307d31
Compare
Co-authored-by: Kruglov Pavel <[email protected]>
| * Quick ready test - you can try this disk, by using these queries (disk has two tables) and this endpoint: | ||
| * | ||
| * ATTACH TABLE contributors UUID 'a563f7d8-fb00-4d50-a563-f7d8fb007d50' (good_person_name String) engine=MergeTree() order by good_person_name settings storage_policy='web'; | ||
| * ATTACH TABLE test UUID '11c7a2f9-a949-4c88-91c7-a2f9a949ec88' (a Int32) engine=MergeTree() order by a settings storage_policy='web'; | ||
| * | ||
| * <storage_configuration> | ||
| * <disks> | ||
| * <web> | ||
| * <type>web</type> | ||
| * <endpoint>https://clickhouse-datasets.s3.yandex.net/kssenii-static-files-disk-test/kssenii-disk-tests/test1/</endpoint> | ||
| * <files_prefix>data</files_prefix> | ||
| * </web> | ||
| * </disks> |
| # Files will be put into /usr/share/nginx/files. | ||
|
|
||
| nginx: | ||
| image: kssenii/nginx-test:1.1 |
There was a problem hiding this comment.
How can I look at Dockerfile for this image?
Maybe it worth to move this image into clickhouse namespace?
There was a problem hiding this comment.
And by the way, maybe official nginx image + file overrides will be enough?
There was a problem hiding this comment.
How can I look at Dockerfile for this image?
Unfortunately I do not have it anymore.
And by the way, maybe official nginx image + file overrides will be enough?
As I remember I tried it when adding these tests, but something did not work and I decided to write my own image, but I do not remember what exactly, it was more than a year ago...
There was a problem hiding this comment.
We have to write new Dockerfile then, put it into ClickHouse repo and upload image to clickhouse namespace
| description.add_options() | ||
| ("help,h", "produce help message") | ||
| ("metadata-path", po::value<std::string>(), "Metadata path (select data_paths from system.tables where name='table_name'") | ||
| ("test-mode", "Use test mode, which will put data on given url via PUT") |
There was a problem hiding this comment.
Maybe http-mode or webdav-mode (or similar) will be a better name?
There was a problem hiding this comment.
Probably, but I named it a test mode because it is very doubtful that somebody will use it as currently it does not even allow credentials for this mode or to replace PUT with POST... So we can replace, but I think then need to make the code more configurable in this place.
6 participants
I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
IDisk interface to store data on web server of static files. Closes #23982.