Feat: Add clickhouse cluster mode setup via docker compose by kavirajk · Pull Request #1749 · ClickHouse/clickhouse-go

kavirajk · 2026-01-06T09:23:17Z

We need cluster mode setup for testing (both manual testing and automated). Both TCP(native) and HTTP

Long term goal is to run all the tests both in single node and multi-node cluster.

Lot's of cluster mode configurations are copied from clickhouse-js client.. But that setup doesn't support TCP load-balancing. I added support for TCP on top of it.

I made sure it works with TCP.

Run the cluster

docker compose -f docker-compose.cluster.yml up

Check 2 nodes cluster with nginx proxy for both HTTP and TCP

$ docker ps --format 'table{{.Names}}\t{{.Ports}}'
NAMES                      PORTS
clickhouse-server-node-1   9009/tcp, 0.0.0.0:9181->9181/tcp, [::]:9181->9181/tcp, 0.0.0.0:8124->8123/tcp, [::]:8124->8123/tcp, 0.0.0.0:9002->9000/tcp, [::]:9002->9000/tcp
clickhouse-server-node-2   9009/tcp, 0.0.0.0:8125->8123/tcp, [::]:8125->8123/tcp, 0.0.0.0:9001->9000/tcp, [::]:9001->9000/tcp, 0.0.0.0:9182->9181/tcp, [::]:9182->9181/tcp
clickhouse-go-nginx        0.0.0.0:8123->8123/tcp, [::]:8123->8123/tcp, 80/tcp, 0.0.0.0:9000->9000/tcp, [::]:9000->9000/tcp

Connect to TCP via proxy

$ clickhouse-client -q 'SELECT cluster,host_name FROM system.clusters FORMAT pretty'
   ┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓
   ┃ cluster      ┃ host_name   ┃
   ┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━┩
1. │ test_cluster │ clickhouse1 │
   ├──────────────┼─────────────┤
2. │ test_cluster │ clickhouse2 │
   └──────────────┴─────────────┘
$

Connect to HTTP via proxy

$ curl "http://localhost:8123" --data-binary "SELECT cluster,host_name FROM system.clusters FORMAT pretty"
   ┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓
   ┃ cluster      ┃ host_name   ┃
   ┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━┩
1. │ test_cluster │ clickhouse1 │
   ├──────────────┼─────────────┤
2. │ test_cluster │ clickhouse2 │
   └──────────────┴─────────────┘

Summary

Checklist

Delete items not relevant to your PR:

Unit and integration tests covering the common scenarios were added
A human-readable description of the changes was provided to include in CHANGELOG
For significant changes, documentation in https://github.com/ClickHouse/clickhouse-docs was updated with further explanations or tutorials

Long term goal is to run all the tests both in single node and multi-node cluster. Signed-off-by: Kaviraj <[email protected]>

chernser · 2026-01-06T16:07:50Z

.docker/nginx/http.conf

+
+server {
+    listen 8123;
+    client_max_body_size 100M;


is this limitation documented? I think it may cause someone investigating "strange" issue if testing big payload.
I would put some readme and text about it. In general it would be good to have a readme about what this docker compose does and what components are present.

If any client try to send request more than configured value, nginix will send the proper error message with the status code (413 request entity too large). Tested it on local by setting 10 bytes client_max_body_size

$ curl -X POST http://localhost:8123 -d "SELECT 1" 1 $ curl -X POST http://localhost:8123 -d "more than 10 bytes of payload" <html> <head><title>413 Request Entity Too Large</title></head> <body> <center><h1>413 Request Entity Too Large</h1></center> <hr><center>nginx/1.23.1</center> </body> </html> $

docker-compose.yml

Copilot

Pull request overview

This PR introduces a Docker Compose-based ClickHouse cluster setup for testing purposes, supporting both TCP (native) and HTTP protocols with load balancing via Nginx.

Key Changes:

Added a new docker-compose.cluster.yml file that defines a 2-node ClickHouse cluster with Nginx as a load balancer
Configured Nginx to provide round-robin load balancing for both HTTP (port 8123) and TCP (port 9000) connections
Created ClickHouse server configuration files for cluster coordination, replication, and Keeper setup

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
docker-compose.yml	Updated to use parameterized ClickHouse version with alpine variant as default
docker-compose.cluster.yml	New cluster configuration defining two ClickHouse nodes and Nginx load balancer
.docker/nginx/tcp.conf	TCP stream configuration for load balancing native ClickHouse protocol
.docker/nginx/nginx.conf	Main Nginx configuration with HTTP and stream modules
.docker/nginx/http.conf	HTTP configuration for load balancing ClickHouse HTTP interface
.docker/clickhouse/users.xml	User configuration with default user and random load balancing
.docker/clickhouse/cluster/server1_config.xml	Configuration for first cluster node including Keeper and replication settings
.docker/clickhouse/cluster/server2_config.xml	Configuration for second cluster node including Keeper and replication settings
.docker/clickhouse/cluster/server1_macros.xml	Macro definitions for first node (cluster, replica, shard identifiers)
.docker/clickhouse/cluster/server2_macros.xml	Macro definitions for second node (cluster, replica, shard identifiers)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-06T16:10:59Z

.docker/clickhouse/cluster/server1_config.xml

+          <port>9100</port>
+        </replica>
+        <replica>
+          <host>clickhouse2</host>
+          <port>9100</port>


The port 9100 specified in the remote_servers configuration does not match the Keeper/Raft port (9181) or the standard TCP port (9000). This appears to be a misconfiguration that would prevent cluster communication. The replica ports should likely be 9000 to match the tcp_port configuration on line 5.

Suggested change

<port>9100</port>

</replica>

<replica>

<host>clickhouse2</host>

<port>9100</port>

<port>9000</port>

</replica>

<replica>

<host>clickhouse2</host>

<port>9000</port>

Copilot · 2026-01-06T16:10:59Z

.docker/clickhouse/cluster/server2_config.xml

+          <port>9100</port>
+        </replica>
+        <replica>
+          <host>clickhouse2</host>
+          <port>9100</port>


The port 9100 specified in the remote_servers configuration does not match the Keeper/Raft port (9181) or the standard TCP port (9000). This appears to be a misconfiguration that would prevent cluster communication. The replica ports should likely be 9000 to match the tcp_port configuration on line 5.

Suggested change

<port>9100</port>

</replica>

<replica>

<host>clickhouse2</host>

<port>9100</port>

<port>9000</port>

</replica>

<replica>

<host>clickhouse2</host>

<port>9000</port>

Copilot · 2026-01-06T16:10:59Z

.docker/clickhouse/cluster/server1_config.xml

+        <port>9100</port>
+      </server>
+      <server>
+        <id>2</id>
+        <hostname>clickhouse2</hostname>
+        <port>9100</port>


The Raft configuration port 9100 does not match the keeper_server tcp_port 9181 configured on line 46. Keeper nodes should communicate using the port specified in keeper_server/tcp_port. Update this to 9181 for proper Keeper cluster communication.

Suggested change

<port>9100</port>

</server>

<server>

<id>2</id>

<hostname>clickhouse2</hostname>

<port>9100</port>

<port>9181</port>

</server>

<server>

<id>2</id>

<hostname>clickhouse2</hostname>

<port>9181</port>

Copilot · 2026-01-06T16:11:00Z

.docker/clickhouse/cluster/server2_config.xml

+        <port>9100</port>
+      </server>
+      <server>
+        <id>2</id>
+        <hostname>clickhouse2</hostname>
+        <port>9100</port>


The Raft configuration port 9100 does not match the keeper_server tcp_port 9181 configured on line 46. Keeper nodes should communicate using the port specified in keeper_server/tcp_port. Update this to 9181 for proper Keeper cluster communication.

Suggested change

<port>9100</port>

</server>

<server>

<id>2</id>

<hostname>clickhouse2</hostname>

<port>9100</port>

<port>9181</port>

</server>

<server>

<id>2</id>

<hostname>clickhouse2</hostname>

<port>9181</port>

Signed-off-by: Kaviraj <[email protected]>

docker-compose.yml

Feat: Add clickhouse cluster mode setup via docker compose

7110584

Long term goal is to run all the tests both in single node and multi-node cluster. Signed-off-by: Kaviraj <[email protected]>

kavirajk requested a review from chernser January 6, 2026 09:34

chernser reviewed Jan 6, 2026

View reviewed changes

docker-compose.yml Outdated Show resolved Hide resolved

chernser requested a review from Copilot January 6, 2026 16:10

Copilot AI reviewed Jan 6, 2026

View reviewed changes

kavirajk mentioned this pull request Jan 12, 2026

feat: client-side load balancing #1744

Closed

3 tasks

update to 25.12

b11f6ed

Signed-off-by: Kaviraj <[email protected]>

chernser reviewed Jan 15, 2026

View reviewed changes

docker-compose.yml Show resolved Hide resolved

Merge branch 'main' into kavirajk/support-cluster-mode

2632d41

kavirajk requested a review from chernser January 15, 2026 13:36

chernser approved these changes Jan 15, 2026

View reviewed changes

kavirajk merged commit 9ca1480 into main Jan 15, 2026
13 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feat: Add clickhouse cluster mode setup via docker compose#1749

Feat: Add clickhouse cluster mode setup via docker compose#1749
kavirajk merged 3 commits intomainfrom
kavirajk/support-cluster-mode

kavirajk commented Jan 6, 2026 •

edited

Loading

Uh oh!

chernser Jan 6, 2026

Uh oh!

kavirajk Jan 15, 2026

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Jan 6, 2026

Uh oh!

Copilot AI Jan 6, 2026

Uh oh!

Copilot AI Jan 6, 2026

Uh oh!

Copilot AI Jan 6, 2026

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

kavirajk commented Jan 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Checklist

Uh oh!

chernser Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

kavirajk Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

kavirajk commented Jan 6, 2026 •

edited

Loading