Skip to content

Conversation

@maflcko
Copy link
Member

@maflcko maflcko commented Dec 9, 2023

This:

  • Removes dead code.
  • Avoids unused copies in some places.
  • Adds copies in other places for safety.

@DrahtBot
Copy link
Contributor

DrahtBot commented Dec 9, 2023

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage

For detailed information about the code coverage, see the test coverage report.

Reviews

See the guideline for information on the review process.

Type Reviewers
ACK ryanofsky, stickies-v, achow101
Concept ACK hebasto
Stale ACK vasild

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

@DrahtBot DrahtBot changed the title refactor: Remove pre-C++20 code, fs::path cleanup refactor: Remove pre-C++20 code, fs::path cleanup Dec 9, 2023
@hebasto
Copy link
Member

hebasto commented Dec 9, 2023

Concept ACK.

This silences the following (clang 18):

common/args.cpp:288:31: error: returning variable 'm_cached_blocks_path' by reference requires holding mutex 'cs_args' [-Werror,-Wthread-safety-reference-return]
 288 |     if (!path.empty()) return path;
     |                               ^

Does it happen on the master branch?

@vasild
Copy link
Contributor

vasild commented Dec 11, 2023

Does it happen on the master branch?

@hebasto, yes, see #28774

Copy link
Contributor

@vasild vasild left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Almost ACK 0d66eea93f1e115b2e9e00ee2e89cd967f826d22, modulo the comment below about the u8string() method.

@DrahtBot DrahtBot requested a review from hebasto December 11, 2023 10:13
MarcoFalke and others added 5 commits December 11, 2023 17:41
Also, add missing includes:

 #include <system_error>  // for error_code
 #include <type_traits>   // for is_same

 #include <cerrno>        // for errno
Treating std::string as UTF-8 is deprecated in std::filesystem::path
since C++20.

However, it makes this codebase easier to read and maintain to retain
the ability for std::string to hold UTF-8.
`ArgsManager::m_cached_blocks_path` is protected by
`ArgsManager::cs_args` and returning a reference to it after releasing
the mutex is unsafe.

To resolve this, return a copy of the path. This has some performance
penalty which is presumably ok, given that paths are a few 100s bytes
at most and `GetBlocksDirPath()` is not called often.

This silences the following (clang 18):

```
common/args.cpp:288:31: error: returning variable 'm_cached_blocks_path' by reference requires holding mutex 'cs_args' [-Werror,-Wthread-safety-reference-return]
  288 |     if (!path.empty()) return path;
      |                               ^
```

Do the same with
`ArgsManager::GetDataDir()`,
`ArgsManager::GetDataDirBase()` and
`ArgsManager::GetDataDirNet()`.
Copy link
Contributor

@vasild vasild left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 856c887

I realized that this discussion #29040 (comment) is not a blocker for this PR because it merely removes a comment and changes const auto to const u8string which is noop, just a style thing.

@fanquake fanquake requested a review from ryanofsky December 13, 2023 10:26
Copy link
Contributor

@stickies-v stickies-v left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 856c887

Copy link
Contributor

@ryanofsky ryanofsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review ACK 856c887

I suggested a scripted diff followup, but all these changes seem good, and the followup could happen later. I am also a little disturbed that 856c887 doesn't begin with fa but I think I can get over it.

@stickies-v
Copy link
Contributor

but I think I can get over it.

🎅 🎄

@maflcko
Copy link
Member Author

maflcko commented Dec 14, 2023

but I think I can get over it.

🎅 🎄

It wasn't written by me, so I not setting the prefix. This makes it easier for me to (fuzzily) recall which commits were authored by me and whose weren't, by just looking at the prefix.

@maflcko
Copy link
Member Author

maflcko commented Dec 14, 2023

I pushed a commit, but it is not a scripted-diff.

Copy link
Contributor

@ryanofsky ryanofsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review ACK facdafaea8cfc2a37e55173d0500794c80b28f7f.

Only change since last review is a new commit renaming the fs::path::u8string method.

Copy link
Contributor

@vasild vasild left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK facdafaea8cfc2a37e55173d0500794c80b28f7f

To verify that nothing has been forgotten in the last commit which renames fs::path::u8string() to fs::path::utf8string() I added std::u8string u8string() const = delete;. The only call to that function is in BOOST_CHECK(fs::path(str8).u8string() == str8); which I guess is intentional.

@maflcko
Copy link
Member Author

maflcko commented Dec 14, 2023

To verify that nothing has been forgotten in the last commit which renames fs::path::u8string() to fs::path::utf8string() I added std::u8string u8string() const = delete;

This should not be needed, because the two functions return different and incompatible types, so any oversight should result in a compile failure as-is.

@sipa
Copy link
Member

sipa commented Dec 14, 2023

@vasild FWIW, Bitcoin Core used to use json-spirit before the introduction of UniValue. The motivating argument was being able to access the string representation of numeric value (because when amounts are passed as JSON "numbers", we don't want to roundtrip through double before converting to CAmount).

Copy link
Contributor

@ryanofsky ryanofsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review ACK 6666713. Just documentation change since last review.

To clarify and clean things up more, we could also document the fs::path::utf8string() method and fs::u8path() function as being inverses of each other, and maybe rename fs::u8path to fs::utf8path for consistency. I don't think either of these things are necessary though, just thoughts for improvement.

@DrahtBot DrahtBot requested a review from vasild December 14, 2023 15:46
@maflcko
Copy link
Member Author

maflcko commented Dec 14, 2023

maybe rename fs::u8path to fs::utf8path for consistency

I think it is nice that it shadows the deprecated "constructor", but I am happy to review a follow-up. (Can be a scripted-diff, because no use of the deprecated u8path remain in the code)

Copy link
Contributor

@stickies-v stickies-v left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

re-ACK 6666713

@achow101
Copy link
Member

ACK 6666713

@achow101 achow101 merged commit 1b2dedb into bitcoin:master Dec 14, 2023
@maflcko maflcko deleted the 2312-fs-cleanup- branch December 14, 2023 21:51
achow101 added a commit that referenced this pull request Jan 23, 2024
… releasing the mutex that guards it

32a9f13 wallet: avoid returning a reference to vMasterKey after releasing the mutex that guards it (Vasil Dimov)

Pull request description:

  `CWallet::GetEncryptionKey()` would return a reference to the internal
  `CWallet::vMasterKey`, guarded by `CWallet::cs_wallet`, which is unsafe.

  Returning a copy would be a shorter solution, but could have security
  implications of the master key remaining somewhere in the memory even
  after `CWallet::Lock()` (the current calls to
  `CWallet::GetEncryptionKey()` are safe, but that is not future proof).

  So, instead of `EncryptSecret(m_storage.GetEncryptionKey(), ...)`
  change the `GetEncryptionKey()` method to provide the encryption
  key to a given callback:
  `m_storage.WithEncryptionKey([](const CKeyingMaterial& k) { EncryptSecret(k, ...); })`

  This silences the following (clang 18):

  ```
  wallet/wallet.cpp:3520:12: error: returning variable 'vMasterKey' by reference requires holding mutex 'cs_wallet' [-Werror,-Wthread-safety-reference-return]
   3520 |     return vMasterKey;
        |            ^
  ```

  ---
  _Previously this PR modified both ArgsManager and wallet code. But the ArgsManager commit 856c887 was merged in #29040 so now this only affects wallet code. The previous PR description was:_

  Avoid this unsafe pattern from `ArgsManager` and `CWallet`:

  ```cpp
  class A
  {
      Mutex mutex;
      Foo member GUARDED_BY(mutex);
      const Foo& Get()
      {
          LOCK(mutex);
          return member;
      } // callers of `Get()` will have access to `member` without owning the mutex.
  ```

ACKs for top commit:
  achow101:
    ACK 32a9f13
  ryanofsky:
    Code review ACK 32a9f13. This seems like a potentially real race condition, and the fix here is pretty simple.
  furszy:
    ACK 32a9f13

Tree-SHA512: 133da84691642afc1a73cf14ad004a7266cb4be1a6a3ec634d131dca5dbcdef52522c1d5eb04f5b6c4e06e1fc3e6ac57315f8fe1e207b464ca025c2b4edefdc1
knst added a commit to knst/dash that referenced this pull request Oct 12, 2024
…h cleanup

It fixes return reference to const variable under mutex which is not a good idea
knst pushed a commit to knst/dash that referenced this pull request Oct 12, 2024
…y after releasing the mutex that guards it

32a9f13 wallet: avoid returning a reference to vMasterKey after releasing the mutex that guards it (Vasil Dimov)

Pull request description:

  `CWallet::GetEncryptionKey()` would return a reference to the internal
  `CWallet::vMasterKey`, guarded by `CWallet::cs_wallet`, which is unsafe.

  Returning a copy would be a shorter solution, but could have security
  implications of the master key remaining somewhere in the memory even
  after `CWallet::Lock()` (the current calls to
  `CWallet::GetEncryptionKey()` are safe, but that is not future proof).

  So, instead of `EncryptSecret(m_storage.GetEncryptionKey(), ...)`
  change the `GetEncryptionKey()` method to provide the encryption
  key to a given callback:
  `m_storage.WithEncryptionKey([](const CKeyingMaterial& k) { EncryptSecret(k, ...); })`

  This silences the following (clang 18):

  ```
  wallet/wallet.cpp:3520:12: error: returning variable 'vMasterKey' by reference requires holding mutex 'cs_wallet' [-Werror,-Wthread-safety-reference-return]
   3520 |     return vMasterKey;
        |            ^
  ```

  ---
  _Previously this PR modified both ArgsManager and wallet code. But the ArgsManager commit bitcoin@856c887 was merged in bitcoin#29040 so now this only affects wallet code. The previous PR description was:_

  Avoid this unsafe pattern from `ArgsManager` and `CWallet`:

  ```cpp
  class A
  {
      Mutex mutex;
      Foo member GUARDED_BY(mutex);
      const Foo& Get()
      {
          LOCK(mutex);
          return member;
      } // callers of `Get()` will have access to `member` without owning the mutex.
  ```

ACKs for top commit:
  achow101:
    ACK 32a9f13
  ryanofsky:
    Code review ACK 32a9f13. This seems like a potentially real race condition, and the fix here is pretty simple.
  furszy:
    ACK 32a9f13

Tree-SHA512: 133da84691642afc1a73cf14ad004a7266cb4be1a6a3ec634d131dca5dbcdef52522c1d5eb04f5b6c4e06e1fc3e6ac57315f8fe1e207b464ca025c2b4edefdc1
PastaPastaPasta added a commit to dashpay/dash that referenced this pull request Oct 21, 2024
…itcoin#28774 - lock annotations

1db9e6a Merge bitcoin#19979: Replace LockAssertion with AssertLockHeld, remove LockAssertion (MarcoFalke)
d0a4198 Merge bitcoin#21598: refactor: Remove negative lock annotations from globals (MarcoFalke)
55114a6 Merge bitcoin#19668: Do not hide compile-time thread safety warnings (MarcoFalke)
898282d Merge bitcoin#28774: wallet: avoid returning a reference to vMasterKey after releasing the mutex that guards it (Ava Chow)
6d45284 fix: add missing lock annotation for ContextualCheckBlock and related TODO (Konstantin Akimov)
846ebab fix: fixes for orphange's locks and cs_main (Konstantin Akimov)
002580c fix: add ignoring safety-annotation for llmq/signing_shares due to template lambdas (Konstantin Akimov)
502d6ae fix: add multiple missing annotation about locks for dash specific code (Konstantin Akimov)
a219a8b partial Merge bitcoin#29040: refactor: Remove pre-C++20 code, fs::path cleanup (Konstantin Akimov)
042e8a4 fix: drop unneded lock assert in net.cpp for m_nodes_mutex (Konstantin Akimov)

Pull request description:

  ## What was done?
  These 4 backports improve noticeable implementation of thread-safety analysis by moving many possible warnings to compilation level.
  There's a lot of fixes for annotations after that!

  ## How Has This Been Tested?
  Build with clang:
  ```
  CC=clang CXX=clang++ ./configure --prefix=$(pwd)/depends/x86_64-pc-linux-gnu --enable-suppress-external-warnings   --enable-debug  --enable-stacktraces --enable-werror --enable-crash-hooks --enable-maintainer-mode
  ```

  ## Breaking Changes
  n/a

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas
  - [x] I have added or updated relevant unit/integration/functional/e2e tests
  - [x] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone

ACKs for top commit:
  UdjinM6:
    utACK 1db9e6a
  PastaPastaPasta:
    utACK 1db9e6a

Tree-SHA512: 42a73e423f311689babc366b58a92f024556cf63af93357cfa0715c471efc1491d02ed9e7ba19b23efcdf423ab5bd051a24da2ad64c32e4b6db132a05f6669c1
@bitcoin bitcoin locked and limited conversation to collaborators Dec 13, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants