Skip to content

Adding "use 'initial_user' option" to user specification in cluster definition #6843

@243f6a8885a308d313198a2e037

Description

Use case
We use max_memory_usage_for_user option to limit the memory usage in order not to allow a single user to occupy a large amount of memory (to maintain availability).
However, the cluster uses another user, defined in cluster definition, instead of query initial_user, therefore the memory limitation only works for the initial server.

https://clickhouse.yandex/docs/en/operations/table_engines/distributed/

Describe the solution you'd like
One solution to this problem is to add a boolean option, use_initial_user_if_possible under cluster name, which means "use initial_user to connect to other shards if possible."
It can be something like:

 <remote_servers>
     <logs>
+        <!-- Optional. Use initial_user (if possible) instead of user specified here. Default: false (use specified user) -->
+        <use_initial_user_if_possible>true</use_initial_user_if_possible>
         <shard>
             <!-- Optional. Shard weight when writing data. Default: 1. -->
             <weight>1</weight>
 ...

pros:

  • can be used with little change in configuration

cons:

  • how can clickhouse-server know the password for initial_user? (it may be hashed in users.xml)
    • simply "use specified user/pass" when the password of initial_user is hashed in users.xml, otherwise use initial_user with its password, can be possible.

Describe alternatives you've considered
I have no better ideas.
(Writing down the "if-else" statements that determine which user to use in XML can be also possible, which seems much more complex.)

Additional context
Many batch processes access to our ClickHouse cluster.
We want to reject any memory-consuming (>32GiB) operations/users because such query can damage the availability of other users/batches.
However, we currently can only limit the memory_usage on the initial server, and usage on other shards are not limited (unless that user also connects to it directly).

Metadata

Metadata

Assignees

No one assigned

    Labels

    comp-distributedDistributed table engine & query routing across shards (sharding/load balancing).comp-rbacAuthorization: roles, grants, quotas, row-level security, access checks.feature

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions