Skip to content

Commit 0b11766

Browse files
nickitatnickitat
authored
Merge branch 'master' into fix_setting
2 parents 6983cc0 + 0f1aece commit 0b11766

File tree

220 files changed

+3599
-1632
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

220 files changed

+3599
-1632
lines changed

‎CHANGELOG.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@
2828
* Add `_etag` virtual column for S3 table engine. Fixes [#65312](https://github.com/ClickHouse/ClickHouse/issues/65312). [#65386](https://github.com/ClickHouse/ClickHouse/pull/65386) ([skyoct](https://github.com/skyoct)).
2929
* Added a tagging (namespace) mechanism for the query cache. The same queries with different tags are considered different by the query cache. Example: `SELECT 1 SETTINGS use_query_cache = 1, query_cache_tag = 'abc'` and `SELECT 1 SETTINGS use_query_cache = 1, query_cache_tag = 'def'` now create different query cache entries. [#68235](https://github.com/ClickHouse/ClickHouse/pull/68235) ([sakulali](https://github.com/sakulali)).
3030
* Support more variants of JOIN strictness (`LEFT/RIGHT SEMI/ANTI/ANY JOIN`) with inequality conditions which involve columns from both left and right table. e.g. `t1.y < t2.y` (see the setting `allow_experimental_join_condition`). [#64281](https://github.com/ClickHouse/ClickHouse/pull/64281) ([lgbo](https://github.com/lgbo-ustc)).
31-
* Intrpret Hive-style partitioning for different engines (`File`, `URL`, `S3`, `AzureBlobStorage`, `HDFS`). Hive-style partitioning organizes data into partitioned sub-directories, making it efficient to query and manage large datasets. Currently, it only creates virtual columns with the appropriate name and data. The follow-up PR will introduce the appropriate data filtering (performance speedup). [#65997](https://github.com/ClickHouse/ClickHouse/pull/65997) ([Yarik Briukhovetskyi](https://github.com/yariks5s)).
31+
* Interpret Hive-style partitioning for different engines (`File`, `URL`, `S3`, `AzureBlobStorage`, `HDFS`). Hive-style partitioning organizes data into partitioned sub-directories, making it efficient to query and manage large datasets. Currently, it only creates virtual columns with the appropriate name and data. The follow-up PR will introduce the appropriate data filtering (performance speedup). [#65997](https://github.com/ClickHouse/ClickHouse/pull/65997) ([Yarik Briukhovetskyi](https://github.com/yariks5s)).
3232
* Add function `printf` for Spark compatiability (but you can use the existing `format` function). [#66257](https://github.com/ClickHouse/ClickHouse/pull/66257) ([李扬](https://github.com/taiyang-li)).
3333
* Add options `restore_replace_external_engines_to_null` and `restore_replace_external_table_functions_to_null` to replace external engines and table_engines to `Null` engine that can be useful for testing. It should work for RESTORE and explicit table creation. [#66536](https://github.com/ClickHouse/ClickHouse/pull/66536) ([Ilya Yatsishin](https://github.com/qoega)).
3434
* Added support for reading `MULTILINESTRING` geometry in `WKT` format using function `readWKTLineString`. [#67647](https://github.com/ClickHouse/ClickHouse/pull/67647) ([Jacob Reckhard](https://github.com/jacobrec)).

‎README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -42,8 +42,6 @@ Keep an eye out for upcoming meetups and events around the world. Somewhere else
4242

4343
Upcoming meetups
4444

45-
* [Bangalore Meetup](https://www.meetup.com/clickhouse-bangalore-user-group/events/303208274/) - September 18
46-
* [Tel Aviv Meetup](https://www.meetup.com/clickhouse-meetup-israel/events/303095121) - September 22
4745
* [Jakarta Meetup](https://www.meetup.com/clickhouse-indonesia-user-group/events/303191359/) - October 1
4846
* [Singapore Meetup](https://www.meetup.com/clickhouse-singapore-meetup-group/events/303212064/) - October 3
4947
* [Madrid Meetup](https://www.meetup.com/clickhouse-spain-user-group/events/303096564/) - October 22
@@ -67,6 +65,8 @@ Recently completed meetups
6765
* [Chicago Meetup (Jump Capital)](https://lu.ma/43tvmrfw) - September 12
6866
* [London Meetup](https://www.meetup.com/clickhouse-london-user-group/events/302977267) - September 17
6967
* [Austin Meetup](https://www.meetup.com/clickhouse-austin-user-group/events/302558689/) - September 17
68+
* [Bangalore Meetup](https://www.meetup.com/clickhouse-bangalore-user-group/events/303208274/) - September 18
69+
* [Tel Aviv Meetup](https://www.meetup.com/clickhouse-meetup-israel/events/303095121) - September 22
7070

7171
## Recent Recordings
7272
* **Recent Meetup Videos**: [Meetup Playlist](https://www.youtube.com/playlist?list=PL0Z2YDlm0b3iNDUzpY1S3L_iV4nARda_U) Whenever possible recordings of the ClickHouse Community Meetups are edited and presented as individual talks. Current featuring "Modern SQL in 2023", "Fast, Concurrent, and Consistent Asynchronous INSERTS in ClickHouse", and "Full-Text Indices: Design and Experiments"

‎cmake/autogenerated_versions.txt‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -2,11 +2,11 @@
22

33
# NOTE: VERSION_REVISION has nothing common with DBMS_TCP_PROTOCOL_VERSION,
44
# only DBMS_TCP_PROTOCOL_VERSION should be incremented on protocol changes.
5-
SET(VERSION_REVISION 54490)
5+
SET(VERSION_REVISION 54491)
66
SET(VERSION_MAJOR 24)
7-
SET(VERSION_MINOR 9)
7+
SET(VERSION_MINOR 10)
88
SET(VERSION_PATCH 1)
9-
SET(VERSION_GITHASH e02b434d2fc0c4fbee29ca675deab7474d274608)
10-
SET(VERSION_DESCRIBE v24.9.1.1-testing)
11-
SET(VERSION_STRING 24.9.1.1)
9+
SET(VERSION_GITHASH b12a367741812f9e5fe754d19ebae600e2a2614c)
10+
SET(VERSION_DESCRIBE v24.10.1.1-testing)
11+
SET(VERSION_STRING 24.10.1.1)
1212
# end of autochange

‎contrib/postgres‎

‎docker/docs/builder/Dockerfile‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,11 @@
11
# docker build -t clickhouse/docs-builder .
2+
FROM golang:alpine AS htmltest-builder
3+
4+
ARG HTMLTEST_VERSION=0.17.0
5+
6+
RUN CGO_ENABLED=0 go install github.com/wjdp/htmltest@v${HTMLTEST_VERSION} \
7+
&& mv "${GOPATH}/bin/htmltest" /usr/bin/htmltest
8+
29
# nodejs 17 prefers ipv6 and is broken in our environment
310
FROM node:16-alpine
411

@@ -17,6 +24,13 @@ RUN yarn config set registry https://registry.npmjs.org \
1724
&& yarn install \
1825
&& yarn cache clean
1926

27+
ENV HOME /opt/clickhouse-docs
28+
29+
RUN mkdir /output_path \
30+
&& chmod -R a+w . /output_path \
31+
&& git config --global --add safe.directory /opt/clickhouse-docs
32+
2033
COPY run.sh /run.sh
34+
COPY --from=htmltest-builder /usr/bin/htmltest /usr/bin/htmltest
2135

2236
ENTRYPOINT ["/run.sh"]

‎docs/.htmltest.yml‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
DirectoryPath: /test
2+
IgnoreDirectoryMissingTrailingSlash: true
3+
CheckExternal: false
4+
CheckInternal: false
5+
CheckInternalHash: true
6+
CheckMailto: false
7+
IgnoreAltMissing: true
8+
IgnoreEmptyHref: true
9+
IgnoreInternalEmptyHash: true

‎docs/en/engines/table-engines/integrations/postgresql.md‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,20 @@
11
---
22
slug: /en/engines/table-engines/integrations/postgresql
3+
title: PostgreSQL Table Engine
34
sidebar_position: 160
45
sidebar_label: PostgreSQL
56
---
67

7-
# PostgreSQL
8-
9-
The PostgreSQL engine allows to perform `SELECT` and `INSERT` queries on data that is stored on a remote PostgreSQL server.
8+
The PostgreSQL engine allows `SELECT` and `INSERT` queries on data stored on a remote PostgreSQL server.
109

1110
:::note
1211
Currently, only PostgreSQL versions 12 and up are supported.
1312
:::
1413

14+
:::note Replicating or migrating Postgres data with with PeerDB
15+
> In addition to the Postgres table engine, you can use [PeerDB](https://docs.peerdb.io/introduction) by ClickHouse to set up a continuous data pipeline from Postgres to ClickHouse. PeerDB is a tool designed specifically to replicate data from Postgres to ClickHouse using change data capture (CDC).
16+
:::
17+
1518
## Creating a Table {#creating-a-table}
1619

1720
``` sql

‎docs/en/engines/table-engines/integrations/s3queue.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -191,6 +191,15 @@ For 'Ordered' mode. Available since `24.6`. If there are several replicas of S3Q
191191
Engine supports all s3 related settings. For more information about S3 settings see [here](../../../engines/table-engines/integrations/s3.md).
192192

193193

194+
## S3Queue Ordered mode {#ordered-mode}
195+
196+
`S3Queue` processing mode allows to store less metadata in ZooKeeper, but has a limitation that files, which added later by time, are required to have alphanumerically bigger names.
197+
198+
`S3Queue` `ordered` mode, as well as `unordered`, supports `(s3queue_)processing_threads_num` setting (`s3queue_` prefix is optional), which allows to control number of threads, which would do processing of `S3` files locally on the server.
199+
In addition, `ordered` mode also introduces another setting called `(s3queue_)buckets` which means "logical threads". It means that in distributed scenario, when there are several servers with `S3Queue` table replicas, where this setting defines the number of processing units. E.g. each processing thread on each `S3Queue` replica will try to lock a certain `bucket` for processing, each `bucket` is attributed to certain files by hash of the file name. Therefore, in distributed scenario it is highly recommended to have `(s3queue_)buckets` setting to be at least equal to the number of replicas or bigger. This is fine to have the number of buckets bigger than the number of replicas. The most optimal scenario would be for `(s3queue_)buckets` setting to equal a multiplication of `number_of_replicas` and `(s3queue_)processing_threads_num`.
200+
The setting `(s3queue_)processing_threads_num` is not recommended for usage before version `24.6`.
201+
The setting `(s3queue_)buckets` is available starting with version `24.6`.
202+
194203
## Description {#description}
195204

196205
`SELECT` is not particularly useful for streaming import (except for debugging), because each file can be imported only once. It is more practical to create real-time threads using [materialized views](../../../sql-reference/statements/create/view.md). To do this:

‎docs/en/engines/table-engines/mergetree-family/graphitemergetree.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -79,7 +79,7 @@ All of the parameters excepting `config_section` have the same meaning as in `Me
7979

8080
## Rollup Configuration {#rollup-configuration}
8181

82-
The settings for rollup are defined by the [graphite_rollup](../../../operations/server-configuration-parameters/settings.md#server_configuration_parameters-graphite) parameter in the server configuration. The name of the parameter could be any. You can create several configurations and use them for different tables.
82+
The settings for rollup are defined by the [graphite_rollup](../../../operations/server-configuration-parameters/settings.md#graphite) parameter in the server configuration. The name of the parameter could be any. You can create several configurations and use them for different tables.
8383

8484
Rollup configuration structure:
8585

‎docs/en/engines/table-engines/mergetree-family/mergetree.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -710,7 +710,7 @@ Data part is the minimum movable unit for `MergeTree`-engine tables. The data be
710710
### Terms {#terms}
711711

712712
- Disk — Block device mounted to the filesystem.
713-
- Default disk — Disk that stores the path specified in the [path](/docs/en/operations/server-configuration-parameters/settings.md/#server_configuration_parameters-path) server setting.
713+
- Default disk — Disk that stores the path specified in the [path](/docs/en/operations/server-configuration-parameters/settings.md/#path) server setting.
714714
- Volume — Ordered set of equal disks (similar to [JBOD](https://en.wikipedia.org/wiki/Non-RAID_drive_architectures)).
715715
- Storage policy — Set of volumes and the rules for moving data between them.
716716

0 commit comments

Comments
 (0)