Skip to content

Added dedicated user for background operations#93905

Merged
arsenmuk merged 4 commits intomasterfrom
arsenmuk/i91845_dedicated_user_for_background_operations
Jan 28, 2026
Merged

Added dedicated user for background operations#93905
arsenmuk merged 4 commits intomasterfrom
arsenmuk/i91845_dedicated_user_for_background_operations

Conversation

@arsenmuk
Copy link
Copy Markdown
Member

@arsenmuk arsenmuk commented Jan 12, 2026

Changelog category (leave one):

  • Improvement

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Background operations (Mutate, Merge) can now be configured independently via 'background' profile. Previously such operations shared settings with regular queries via 'default' profile.

Closes #91845
Reverts #64456 as no longer needed.

Documentation entry for user-facing changes

  • Documentation is written (mandatory for new features)

@clickhouse-gh
Copy link
Copy Markdown
Contributor

clickhouse-gh bot commented Jan 12, 2026

Workflow [PR], commit [9e43064]

Summary:

job_name test_name status info comment
Integration tests (amd_asan, targeted) failure
test_ttl_move/test.py::test_disabled_ttl_move_on_insert[volume-2-10] FAIL cidb
test_ttl_move/test.py::test_disabled_ttl_move_on_insert[disk-7-10] FAIL cidb
test_ttl_move/test.py::test_disabled_ttl_move_on_insert[disk-10-10] FAIL cidb
test_ttl_move/test.py::test_disabled_ttl_move_on_insert[replicated_volume-2-10] FAIL cidb
test_ttl_move/test.py::test_alter_with_merge_work[mt_work-6-10] FAIL cidb
test_ttl_move/test.py::test_disabled_ttl_move_on_insert[replicated_disk-8-10] FAIL cidb
test_ttl_move/test.py::test_disabled_ttl_move_on_insert[replicated_volume-6-10] FAIL cidb
test_ttl_move/test.py::test_disabled_ttl_move_on_insert[replicated_disk-3-10] FAIL cidb
test_ttl_move/test.py::test_alter_with_merge_work[mt_work-9-10] FAIL cidb
Upgrade check (amd_tsan) failure
BuzzHouse (amd_debug) error

@kssenii kssenii self-assigned this Jan 12, 2026
@arsenmuk arsenmuk force-pushed the arsenmuk/i91845_dedicated_user_for_background_operations branch from 79610db to 70b2d49 Compare January 12, 2026 18:18
@arsenmuk arsenmuk changed the title Added context for background operations Added dedicated user for background operations Jan 12, 2026
@clickhouse-gh clickhouse-gh bot added the pr-improvement Pull request with some product improvements label Jan 12, 2026
@arsenmuk
Copy link
Copy Markdown
Member Author

Couple of TODOs.

  1. Need to restrict background profile from potentially used to connect to server. The following draft works preliminarily, I'm assessing whether it's enough or something else is needed:
void Context::setUser(const UUID & user_id_, const std::vector<UUID> & external_roles_)
{
    /// Prepare lists of user's profiles, constraints, settings, roles.
    /// NOTE: AccessControl::read<User>() and other AccessControl's functions may require some IO work,
    /// so Context::getLocalLock() and Context::getGlobalLock() must be unlocked while we're doing this.

    auto & access_control = getAccessControl();
    auto user = access_control.read<User>(user_id_);

    auto default_roles = user->granted_roles.findGranted(user->default_roles);
    auto enabled_roles = access_control.getEnabledRolesInfo(default_roles, {});
    auto enabled_profiles = access_control.getEnabledSettingsInfo(user_id_, user->settings, enabled_roles->enabled_roles, enabled_roles->settings_from_enabled_roles);

+    std::unordered_set<String> system_profiles = {
+        getGlobalContextInstance()->shared->system_profile_name,
+        getGlobalContextInstance()->shared->background_profile_name,
+        getGlobalContextInstance()->shared->buffer_profile_name
+    };
+
+    for (const auto & [profile_id, profile_name] : enabled_profiles->names_of_profiles)
+    {
+         if (system_profiles.contains(profile_name))
+         {
+            throw Exception(
+                ErrorCodes::ACCESS_DENIED,
+                "Profile '{}' is a system profile and cannot be used for user connections.",
+                profile_name);
+         }
+    }
  1. There was a kinda similar change - Don't propagate user settings for background ops #64456 - which potentially becomes rudimentary with this change. I'm assessing whether this should be addressed within the same PR or not:
Context::isBackgroundOperationContext()
Context::setBackgroundOperationTypeForContext()

@arsenmuk arsenmuk force-pushed the arsenmuk/i91845_dedicated_user_for_background_operations branch 6 times, most recently from e7669ac to 15f76d2 Compare January 20, 2026 15:19
@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Jan 20, 2026

CLA assistant check
All committers have signed the CLA.

@arsenmuk arsenmuk force-pushed the arsenmuk/i91845_dedicated_user_for_background_operations branch from 15f76d2 to b5d8bef Compare January 20, 2026 15:22
@arsenmuk
Copy link
Copy Markdown
Member Author

arsenmuk commented Jan 20, 2026

Couple of TODOs.

  1. Need to restrict background profile from potentially used to connect to server. The following draft works preliminarily, I'm assessing whether it's enough or something else is needed:

The settings are given in the profile 'background'. Connectivity using such profile is possible by explicitly setting up a user. So restriction is not necessary.

  1. There was a kinda similar change - Don't propagate user settings for background ops #64456 - which potentially becomes rudimentary with this change. I'm assessing whether this should be addressed within the same PR or not:

done

@arsenmuk arsenmuk force-pushed the arsenmuk/i91845_dedicated_user_for_background_operations branch from b5d8bef to 72d083d Compare January 20, 2026 15:50
@arsenmuk arsenmuk force-pushed the arsenmuk/i91845_dedicated_user_for_background_operations branch from 72d083d to 80c824c Compare January 20, 2026 19:38
@arsenmuk arsenmuk marked this pull request as ready for review January 22, 2026 09:00
@arsenmuk arsenmuk marked this pull request as draft January 22, 2026 09:04
@arsenmuk arsenmuk force-pushed the arsenmuk/i91845_dedicated_user_for_background_operations branch from a792ec3 to a31c632 Compare January 22, 2026 09:07
@arsenmuk arsenmuk marked this pull request as ready for review January 22, 2026 09:16
@arsenmuk
Copy link
Copy Markdown
Member Author

Failures of Integration tests (amd_asan, targeted), specifically 'test_ttl_move' and its friends, is another flavour of #85011.

What happens is:

  • first this test failed due to natural reasons at early implementation which was completely broken
  • then this test became a part of 'targeted' suite as suspicious, and it got hammered at each execution thoroughly, which exposed the original issue with it
  • the hypothesis is that the timings shift under the pressure at 'targeted' execution, and fire more often

For this PR we assume it as flaky and ignore it. To prove that I drafted #94853 with exactly same code, but in a clean state - those tests do not fail there.

As for further actions, I self-assigned to #85011 and modified my draft #94853 to actually fix the flaky test.

bool is_background_operation = false;

inline static ContextPtr global_context_instance;
inline static ContextPtr background_context_instance; /// Background context is a global concept (kinda singletone for now), and this is a actual place to own it.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
inline static ContextPtr background_context_instance; /// Background context is a global concept (kinda singletone for now), and this is a actual place to own it.
inline static ContextPtr background_context_instance; /// Background context is a global concept (kinda singletone for now), and this is an actual place to own it.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd not use slang, e.g. kinda

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Comment on lines +65 to +66
with pytest.raises(QueryRuntimeException):
query()
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO better to use assert "expected error message" in node.query_and_get_error(query), but not important

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the hint. The test is now updated to incorporate that

@kssenii
Copy link
Copy Markdown
Member

kssenii commented Jan 27, 2026

BTW I think would be convenient to later add something like this for background streaming operations (like s3queue, kafka, etc).

@arsenmuk arsenmuk added this pull request to the merge queue Jan 28, 2026
Merged via the queue into master with commit 3ad2858 Jan 28, 2026
130 of 134 checks passed
@arsenmuk arsenmuk deleted the arsenmuk/i91845_dedicated_user_for_background_operations branch January 28, 2026 13:09
@robot-ch-test-poll1 robot-ch-test-poll1 added the pr-synced-to-cloud The PR is synced to the cloud repo label Jan 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-improvement Pull request with some product improvements pr-synced-to-cloud The PR is synced to the cloud repo

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dedicated user for background operations

4 participants