Skip to content

fix double slashed paths in DatabaseOrdinary by switching to std::filesystem::path#11900

Merged
tavplubix merged 6 commits intoClickHouse:masterfrom
bharatnc:ncb/fix-file-path
Jun 26, 2020
Merged

fix double slashed paths in DatabaseOrdinary by switching to std::filesystem::path#11900
tavplubix merged 6 commits intoClickHouse:masterfrom
bharatnc:ncb/fix-file-path

Conversation

@bharatnc
Copy link
Copy Markdown
Contributor

@bharatnc bharatnc commented Jun 24, 2020

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Changelog category (leave one):

  • Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Improve path concatenation and fix double slashed paths using std::filesystem::path instead of std::string in DatabaseOrdinary.cpp.

Detailed description / Documentation draft:

The concatenation of path doesn't always happen reliably and can cause outputs which contain double slashed paths: (Referencing from issue #8301 and this PR relates to that)

2019.12.19 17:33:18.401719 [ 1 ] {} <Error> Application: DB::Exception: Cannot parse definition from metadata file /var/lib/clickhouse/metadata/default//aaaa.sql, error: DB::Exception: Syntax error (in file /var/lib/clickhouse/metadata/default//aaaa.sql): failed at position 21 (line 3, col 1): )

To fix this I have chosen std::filesystem::path to concatenate the file paths instead of using std::string.

I have also removed several imports that are not being used. Also clang formatted the file and it looks like that did generate several changes to formatting in DatabaseOrdinary.cpp.

@blinkov blinkov added the pr-improvement Pull request with some product improvements label Jun 24, 2020
@tavplubix tavplubix self-assigned this Jun 24, 2020
std::atomic<size_t> tables_processed{0};

auto startup_one_table = [&](const StoragePtr & table) {
// clang-format off
Copy link
Copy Markdown
Contributor Author

@bharatnc bharatnc Jun 24, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like when I run clang-format, it doesn't respect brace wrapping for lambdas like it does for classes and functions. After adding clang-format off and clang-format on tags and manually formatting these lines, it passes the style checks now: https://clickhouse-test-reports.s3.yandex.net/11900/1244ca562759c4b92a08f4350080cc01567e8c99/style_check.html

Before it failed: https://clickhouse-test-reports.s3.yandex.net/11900/ea40e7b4039f69e8e1cb20ce23039ed0c93bc7ca/style_check.html#fail1

Once I had fixed the formatting, I have removed these tags in the latest commit. I'm not sure if this is specific to my dev environment (using Clion and clang-format).

@tavplubix
Copy link
Copy Markdown
Member

The changes LGTM, but could you please replace boost::filesystem::path with std::filesystem::path? You're right, some parts of boost are widely used in ClickHouse codebase, but there is no usages of boost::filesystem and we prefer std::filesystem::path or Poco::Path (probably will be also replaced with std::filesystem sometime).

@bharatnc bharatnc changed the title fix double slashed paths in DatabaseOrdinary by switching to boost::filesystem fix double slashed paths in DatabaseOrdinary by switching to std::filesystem::path Jun 26, 2020
@bharatnc
Copy link
Copy Markdown
Contributor Author

The changes LGTM, but could you please replace boost::filesystem::path with std::filesystem::path? You're right, some parts of boost are widely used in ClickHouse codebase, but there is no usages of boost::filesystem and we prefer std::filesystem::path or Poco::Path (probably will be also replaced with std::filesystem sometime).

Sure 👍. I've changed to std:: filesystem::path.

@tavplubix tavplubix merged commit dfdfc69 into ClickHouse:master Jun 26, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-improvement Pull request with some product improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants