Skip to content

Untangle setting headers#66404

Merged
Algunenano merged 17 commits intoClickHouse:masterfrom
Algunenano:mergetree_private
Jul 15, 2024
Merged

Untangle setting headers#66404
Algunenano merged 17 commits intoClickHouse:masterfrom
Algunenano:mergetree_private

Conversation

@Algunenano
Copy link
Copy Markdown
Member

@Algunenano Algunenano commented Jul 11, 2024

Changelog category (leave one):

  • Not for changelog (changelog entry is not required)

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

  • Untangle setting headers

Documentation entry for user-facing changes

x

This is a step towards private Setting* objects, which is a step itself towards Setting categories.

Massively reduces the number of affected files when SettingEnums.h / Settings.h / MergeSettings.h are changed. For example changing SettingEnums.h goes from affecting ~2500 files to ~650, and Settings.h goes from ~1500 to ~550. This helps massively developing anything that relates to settings, or adding new settings.

I'm merging this first since it's already affecting 440 files and it's only a refactor so far, but it's better to confirm everything is fine before doing additional, hundreds of, changes on top.

CI Settings (Only check the boxes if you know what you are doing):

  • Allow: All Required Checks
  • Allow: Stateless tests
  • Allow: Stateful tests
  • Allow: Integration Tests
  • Allow: Performance tests
  • Allow: All Builds
  • Allow: batch 1, 2 for multi-batch jobs
  • Allow: batch 3, 4, 5, 6 for multi-batch jobs

  • Exclude: Style check
  • Exclude: Fast test
  • Exclude: All with ASAN
  • Exclude: All with TSAN, MSAN, UBSAN, Coverage
  • Exclude: All with aarch64, release, debug

  • Do not test
  • Woolen Wolfdog
  • Upload binaries for special builds
  • Disable merge-commit
  • Disable CI cache

@robot-ch-test-poll4 robot-ch-test-poll4 added the pr-not-for-changelog This PR should not be mentioned in the changelog label Jul 11, 2024
@robot-ch-test-poll3
Copy link
Copy Markdown
Contributor

robot-ch-test-poll3 commented Jul 11, 2024

This is an automated comment for commit 6e67edf with description of existing statuses. It's updated for the latest CI running

✅ Click here to open a full report in a separate page

Successful checks
Check nameDescriptionStatus
Docs checkBuilds and tests the documentation✅ success
Fast testNormally this is the first check that is ran for a PR. It builds ClickHouse and runs most of stateless functional tests, omitting some. If it fails, further checks are not started until it is fixed. Look at the report to see which tests fail, then reproduce the failure locally as described here✅ success
Style checkRuns a set of checks to keep the code style clean. If some of tests failed, see the related log from the report✅ success

@alexey-milovidov alexey-milovidov self-assigned this Jul 11, 2024
@alexey-milovidov
Copy link
Copy Markdown
Member

The changes LGTM.

@alexey-milovidov
Copy link
Copy Markdown
Member

@Algunenano, what do you think about this? #58797

@Algunenano
Copy link
Copy Markdown
Member Author

@Algunenano, what do you think about this? #58797

I'm planning on fixing it and hiding the object contents completely right after this PR. The idea is that the Settings objects will contain an array of SettingFieldBaseClass and most of the time we'd access a setting by calling getUInt64(settingRef, "string"). I prefer not exposing Settings::max_thread as that allows to hide the implementation completely, it's worse for IDEs but much better when we do any change. With this we could add a new setting, or change it's value/description, compile just one file, link and we're done.

For the other 1%, we could have special functions (we already have some here and there)

@nikitamikhaylov
Copy link
Copy Markdown
Member

@Algunenano getUInt64(settingRef, "string") looks like not a good design, because you can easily make a typo and will debug it for hours. About Settings::max_thread: you will not expose it, it will be completely the same as ErrorCodes or CurrentMetrics, but you will need to export them everywhere.

@Algunenano
Copy link
Copy Markdown
Member Author

Algunenano commented Jul 11, 2024

@Algunenano getUInt64(settingRef, "string") looks like not a good design, because you can easily make a typo and will debug it for hours.
About Settings::max_thread: you will not expose it

We could do for something like (using MergeTreeSettings because it's easier to see):

  • Call
#include <Storages/MergeTree/MergeTreeSettingGetter.h>

namespace MergeTreeSettings
{
extern const int replicated_max_parallel_fetches_for_table;
}

getInt64(settingRef, MergeTreeSettings::replicated_max_parallel_fetches_for_table)
  • Storages/MergeTree/MergeTreeSettingGetter.h: Would just have the forward reference for MergeTreeSettingsStruct and for the getters.

Does that makes more sense?

@nikitamikhaylov
Copy link
Copy Markdown
Member

nikitamikhaylov commented Jul 11, 2024

Yes, exactly. This is what was written in Alexey's issue I suppose 🤣
But wait, why do we need to a forward reference? CurrentMetrics work without it. And you will never modify the header Settings.h afterwards, only Settings.cpp. Yes, it will be harder to develop something around Settings during that time, but will be Ok for the future.

@Algunenano
Copy link
Copy Markdown
Member Author

But wait, why do we need to a forward reference? CurrentMetrics work without it.

We are not passing, copying and modifying CurrentMetrics objects all the time. If everyone knows about Settings, everyone needs to know about BaseSettings and SettingEnums, and again adding a new setting or a value might involve rebuilding thousands of files. Most of the users never hold an object, just a reference, so it's more natural to not expose it.

@Algunenano
Copy link
Copy Markdown
Member Author

Another reason to prefer forward declarations: Many times when we use objects, it's by mistake: 56f497b. I should probably remove getSettingsRef(), make getSettings() return a const reference, and make explicit copies when needed.

@alexey-milovidov
Copy link
Copy Markdown
Member

Yes, I'm in favor of this change. Maybe rename getSettings to getSettingsCopy?

@Algunenano Algunenano added this pull request to the merge queue Jul 12, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jul 12, 2024
@Algunenano Algunenano enabled auto-merge July 12, 2024 14:53
@Algunenano Algunenano added this pull request to the merge queue Jul 12, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jul 12, 2024
@Algunenano Algunenano enabled auto-merge July 12, 2024 17:31
@Algunenano Algunenano added this pull request to the merge queue Jul 15, 2024
Merged via the queue into ClickHouse:master with commit e4f348a Jul 15, 2024
@Algunenano Algunenano deleted the mergetree_private branch July 15, 2024 09:29
@robot-clickhouse-ci-1 robot-clickhouse-ci-1 added the pr-synced-to-cloud The PR is synced to the cloud repo label Jul 15, 2024
baibaichen added a commit to Kyligence/gluten that referenced this pull request Jul 16, 2024
baibaichen added a commit to baibaichen/gluten that referenced this pull request Jul 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-not-for-changelog This PR should not be mentioned in the changelog pr-synced-to-cloud The PR is synced to the cloud repo

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants