Skip to content

Comments

common: add missing include, reduce header dependencies#60253

Merged
yuriw merged 18 commits intoceph:mainfrom
MaxKellermann:includes
Nov 21, 2024
Merged

common: add missing include, reduce header dependencies#60253
yuriw merged 18 commits intoceph:mainfrom
MaxKellermann:includes

Conversation

@MaxKellermann
Copy link
Member

Ceph build times are pretty bad, even for a C++ project; part of that is because Ceph has too much header+template bloat. I have started reducing header dependencies, but found many build failures because #include lines were missing everywhere. This PR starts adding them, to prepare for the actual cleanup.

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)

@MaxKellermann
Copy link
Member Author

"stderr: fatal: unable to access 'https://github.com/ceph/ceph.git/': Could not resolve host: github.com"

sigh ...

@MaxKellermann MaxKellermann force-pushed the includes branch 3 times, most recently from d0a3fff to ee138e7 Compare October 14, 2024 11:07
@MaxKellermann
Copy link
Member Author

Added a patch for radosacl.cc because another #include was missing there which broke the CI build.

#include "include/spinlock.h"
#include "include/scope_guard.h"

#include <iostream>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here

@ronen-fr
Copy link
Contributor

Just to make sure: crimson compiled OK after your changes?

@MaxKellermann
Copy link
Member Author

Just to make sure: crimson compiled OK after your changes?

I don't know what crimson is, and I didn't build all combinations of all Ceph features - I struggled a lot until I could build Ceph at all.

@MaxKellermann
Copy link
Member Author

Google's cppguide forbids forward declarations.

Avoid using forward declarations where possible.

I do not agree with that, but since Ceph mandates the use of Google's cppguide, I should follow it.
Part of this PR violates this guide (and much of existing Ceph code violates it, too).

What shall I do? Remove forward declarations? (From this PR and submit PRs to remove existing forward declarations)

@ronen-fr
Copy link
Contributor

Google's cppguide forbids forward declarations.

Avoid using forward declarations where possible.

I do not agree with that, but since Ceph mandates the use of Google's cppguide, I should follow it. Part of this PR violates this guide (and much of existing Ceph code violates it, too).

What shall I do? Remove forward declarations? (From this PR and submit PRs to remove existing forward declarations)

I do not think we are following this. I, for one, certainly use fwd declarations.

@MaxKellermann
Copy link
Member Author

I do not think we are following this. I, for one, certainly use fwd declarations.

I'll post a PR for CodingStyle explicitly allowing forward declarations. Sounds like it may get accepted. IMHO forward declarations reduce the pain of C++'s unbearable header bloat.

(I wish we could use C++20 modules. They solve this issue once and for all. But modules cause headaches of its own; unfortunately it's an extremely badly designed language feature.)

This header uses `std::string`, but not `std::system_error`.

Signed-off-by: Max Kellermann <[email protected]>
And add a few missing includes.

Signed-off-by: Max Kellermann <[email protected]>
@MaxKellermann
Copy link
Member Author

@ronen-fr all (bogus) CI failures are now fixed, and since Crimson is part of the CI, I suppose it's fine.

@ronen-fr
Copy link
Contributor

ronen-fr commented Nov 5, 2024

I'll post a PR for CodingStyle explicitly allowing forward declarations. Sounds like it may get accepted. IMHO forward declarations reduce the pain of C++'s unbearable header bloat.

Good work! It's great to see that one merged so quickly

#include <cstdint>

#include "common/perf_histogram.h"
#include "include/utime.h"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's the final issue I have with this commit set:
utime_t is an integral part of the interface of the perf-counters.
(and - a random sampling of files that include this header show that almost all of those files make use of 'utime_t').
I am not sure it makes sense to forward-declare it here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The header defines dozens of functions, but only 3 of those have utime_t in their signature. These 3 methods are just one special case, but not what I'd call "integral part".
Look at the heavy dependencies of utime.h - it pulls in the whole encoding/denc library with all its dependencies (= all C++ containers plus a bunch of Boost containers, the buffer library), SubProcess.h (iostreams + sstream), strtol.h (charconv!).

utime.h is one of those headers that gets included everywhere by everybody, implicitly. Yes, it is used often, but those sources that really need it should include it explicitly. Forward-declaring it from perf_counters.h helps identifying those sources that mistakenly forgot to include utime.h. That improves code correctness.

That code correctness is not just a theoretical principle; it helps finding bugs early that may occur in different build-time configurations. Ceph has lots of build failures when certain features are disabled - look at #60547 for just one of many examples (I have a dozen similar patches not-yet-submitted on my stgit stack).

One idea I'm currently exploring is making more features compile-time optional, for example ceph_assert() (see #60160); that reduces executable size (improved CPU cache hit rate, better branch prediction) and reduces runtime overhead (by omitting evaluations that are only useful in debug builds). The perf_counters feature is one such thing that could be made optional, to be disabled when performance is utterly important. Once perf_counters are optional, those missing utime.h includes will become build failures, and more build failures may sneak in at any time. Therefore, reducing header dependencies has practical advantages - it makes the build more robust.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for example ceph_assert() (see #60160);

Not related to the current PR, but still - we now use ceph_assert() with the knowledge that it will
be active in release builds. It would be a bad idea to "disappear" it (using Catch-22 phrasing)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK - that's convincing enough
(although I don't buy in making the per_counters optional. I think there are strong reasons not to do that)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I saw how much performance can be gained by disabling assertions and (debug-)logging, and that's why we will disable both at compile time, and maybe more. If the code is correct, that's easier. Thanks for supporting my fight for better and more correct code!

@ljflores
Copy link
Member

@yuriw yuriw merged commit b6d7491 into ceph:main Nov 21, 2024
@MaxKellermann MaxKellermann deleted the includes branch November 22, 2024 06:42
@MaxKellermann
Copy link
Member Author

Thanks @yuriw; I will rebase & finish #60490 soon. It's a huge one and is a continuation of this PR, with 28% compiler speedup.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants