d3d6a18 docs: Add 371 to bips.md (Andrew Chow)

Pull request description:

ACKs for top commit:
  fanquake:
    ACK d3d6a18

Tree-SHA512: dc25742c4fe46d4bb80023802823e61b8081964c5ccd6627dcbe531727e0442926b402cf909937ccf5713a0d0f917cb0eab2c743330afb57216b54325d7431f0

This is a trivial follow-up to bitcoin#26124.

a9d20ee doc: bump bips.md up-to-date version to v24.0 (Sebastian Falbesoner)

Pull request description:

  This is a trivial follow-up to #26124.

ACKs for top commit:
  jarolrod:
    ACK a9d20ee

Tree-SHA512: 24c17c72498f96f9122d8fb041f1f6f63bd186e25ac3cb5a661bb1993106c6632f5efd95a15d19681004d30d38eca2d2a16b383a7a1f1c3db17f887ae1fcd02a

GitHub-Pull: bitcoin#26124
Rebased-From: d3d6a18

e2e4c29 tests: Test that PSBT_OUT_TAP_TREE is included correctly (Andrew Chow)
4d42c3a psbt: Only include m_tap_tree if it has scripts (Andrew Chow)
d810fde psbt: Change m_tap_tree to store just the tuples (Andrew Chow)
a9419ef tests: Test that PSBT_OUT_TAP_TREE is combined correctly (Andrew Chow)
4abd2ab psbt: Fix merging of m_tap_tree (Andrew Chow)
1390c96 [BugFix]: Do not allow deserializing PSBT with empty PSBT_OUT_TAP_TREE (Jeremy Rubin)
9b438f0 refactor: revert m_next_resend to not be std::atomic (stickies-v)
43ced0b wallet: only update m_next_resend when actually resending (stickies-v)
fc8f2bf refactor: carve out tx resend timer logic into ShouldResend (stickies-v)
a6fb674 refactor: remove unused locks for ResubmitWalletTransactions (stickies-v)
5ad82a0 index: Improve BaseIndex::BlockUntilSyncedToCurrentChain reliability (Ryan Ofsky)
997faf6 contrib: Fix capture_output in getcoins.py (willcl-ark)
7e0bcfb p2p: ProcessHeadersMessage(): fix received_new_header (Larry Ruane)
c97d924 Correct sanity-checking script_size calculation (Pieter Wuille)
da6fba6 docs: Add 371 to bips.md (Andrew Chow)

Pull request description:

  Will collect backports for rc2 as they become available. Currently:
  * #25858
  * #26124
  * #26149
  * #26172
  * #26205
  * #26212
  * #26215

ACKs for top commit:
  dergoegge:
    ACK e2e4c29
  achow101:
    ACK e2e4c29
  instagibbs:
    ACK e2e4c29

Tree-SHA512: b6374fe202561057dbe1430d4c40f06f721eb568f91e7275ae1ee7747edf780ce64620382d13ecc4b9571d931dc25d226af8284987cf35ff6a6182c5f64eb10c

This is a trivial follow-up to bitcoin#26124.

* doc: add info about status code 404 for some rest endpoints

* refactor: remove duplicate code from BlockAssembler

* build: Fix `capnp` package build for Android

* build: Specify native binaries explicitly when building `capnp` package

From `configure --help`:
  --with-external-capnp   use the system capnp binary (or the one specified
                          with $CAPNP) instead of compiling a new one (useful
                          for cross-compiling)

* net: remove useless call to IsReachable() from CConnman::Bind()

`CConnman::Bind()` is called without `BF_EXPLICIT` only when passed
either `0.0.0.0` or `::`. For those addresses `IsReachable()` is always
true (regardless of the `-onlynet=` setting!), meaning that the `if`
condition never evaluates to true.

`IsReachable()` is always true for the "any" IPv4 and IPv6 addresses
because `CNetAddr::GetNetwork()` returns `NET_UNROUTABLE` instead of
`NET_IPV4` or `NET_IPV6` and the network `NET_UNROUTABLE` is always
considered reachable.

It follows that `BF_EXPLICIT` is unnecessary, remove it too.

* sync: simplify MaybeCheckNotHeld() definitions by using a template

Reduce 4 of the `MaybeCheckNotHeld()` definitions to 2 by using a
template. This also makes the function usable for other
[BasicLockable](https://en.cppreference.com/w/cpp/named_req/BasicLockable)
types.

* sync: remove unused template parameter from ::UniqueLock

The template parameter `typename Base = typename Mutex::UniqueLock` is
not used, so remove it. Use internally defined type `Base` to avoid
repetitions of `Mutex::UniqueLock`.

* scripted-diff: Rename time symbols

-BEGIN VERIFY SCRIPT-

 ren() { sed -i "s:\<$1\>:$2:g" $(git grep -l "\<$1\>" ':(exclude)src/versionbits.cpp') ; }

 ren nStart                 time_start
 ren nTimeStart             time_start
 ren nTimeReadFromDiskTotal time_read_from_disk_total
 ren nTimeConnectTotal      time_connect_total
 ren nTimeFlush             time_flush
 ren nTimeChainState        time_chainstate
 ren nTimePostConnect       time_post_connect
 ren nTimeCheck             time_check
 ren nTimeForks             time_forks
 ren nTimeConnect           time_connect
 ren nTimeVerify            time_verify
 ren nTimeUndo              time_undo
 ren nTimeIndex             time_index
 ren nTimeTotal             time_total
 ren nTime1                 time_1
 ren nTime2                 time_2
 ren nTime3                 time_3
 ren nTime4                 time_4
 ren nTime5                 time_5
 ren nTime6                 time_6

 ren nBlocksTotal num_blocks_total

 # Newline after semicolon
 perl -0777 -pi -e 's/; time_connect_total/;\n        time_connect_total/g' src/validation.cpp
 perl -0777 -pi -e 's/; time_/;\n    time_/g'                               src/validation.cpp

-END VERIFY SCRIPT-

* Use steady clock for bench logging

* doc: add historical 0.20.2 release notes

* doc: add historical 0.21.2 release notes

* build: split ARM crc & crypto extension checks

We currently perform the same check twice, to put the same set of flags
in two different variables. Split the checks so we test for crc and crypto
extensions independently.

If we don't want to split, we should just delete the second AX_CHECK_COMPILE_FLAG
check, and set ARM_CRC_CXXFLAGS & ARM_CRC_CXXFLAGS at the same time.

* refactor: Do not discard `try_lock()` return value

Microsoft's C++ Standard Library uses the `[[nodiscard]]` attribute for
`try_lock()`.
See: https://github.com/microsoft/STL/blob/main/stl/inc/mutex

This change allows to drop the current suppression for the warning C4838
and helps to prevent the upcoming warning C4858.
See: microsoft/STL@539c26c

* signet/miner: reduce default interblock interval limit to 30min

Also allow the operator to change it, if desired, without having
to edit the code.

* Squashed 'src/leveldb/' changes from 22f1e4a02f..e2f10b4e47

e2f10b4e47 Merge bitcoin-core/leveldb-subtree#34: win32: fix -Wmissing-field-initializers warnings
12c52b392d win32: fix -Wmissing-field-initializers warnings

git-subtree-dir: src/leveldb
git-subtree-split: e2f10b4e47bc950a81bc96d1c6db3a8048216642

* ci: Bump vcpkg to the latest version `2022.09.27`

Dependency changes (2022.06.16.1 - 2022.09.27):
 - boost 1.79.0#0 -> 1.80.0#0
 - sqlite3 3.37.2#1 -> 3.39.2#0
 - zeromq 4.3.4#5 -> 4.3.4#6

* contrib: Fix capture_output in getcoins.py

Our required Python version 3.6.12 does not support `capture_output` as
a subprocess.run argument; this was added in python 3.7.

We can emulate it by setting stdout and stderr to subprocess.PIPE

* fuzz: Limit outpoints.size in txorphan target to avoid OOM

* refactor: move Boost datetime usage to wallet

This means we don't need datetime in a --disable-wallet build, and it
isn't included in the kernel.

* wallet: Use correct effective value when checking target

* test: Check external coin effective value is used in CoinSelection

* test: Use proper Boost macros instead of assertions

* ci: Run `bench_bitcoin.exe --sanity-check` in "Win64 native" task

Also a better name used for the script as it follows GNU's `make check`.

* doc: bump bips.md up-to-date version to v24.0

This is a trivial follow-up to bitcoin#26124.

* refactor: Drop `owns_lock()` call

Co-authored-by: Vasil Dimov <[email protected]>

* refactor: move DEFAULT_TXINDEX from validation to txindex

* refactor: move DEFAULT_COINSTATSINDEX from validation to coinstatsindex

* refactor: move DEFAULT_BLOCKFILTERINDEX from val to blockfilterindex

* kernel: remove util/bytevectorhash.cpp

* ci: Move `git config` commands into script where they are used

* ci: Use same `merge_script` implementation for Windows as for all

* ci: Remove unused package

Address feedback from https://github.com/bitcoin/bitcoin/pull/24561/files#r985719812

* ci: Allow PIP_PACKAGES on centos

This was added in 7fc5e86 but I can't
see a reason why this should be forbidden.

* test: Remove unused fCheckpointsEnabled from miner_tests

The earliest checkpoint is at height 11111, so this can't possibly have
any impact on this test.

* build, msvc: Enable C4834 warning

https://docs.microsoft.com/en-us/cpp/error-messages/compiler-warnings/c4834

* kernel: move RunCommandParseJSON to its own file

Because libbitcoinkernel does not include this new object, this has the
side-effect of eliminating the unnecessary boost::process dependency.

* ci: Workaround Windows filesystem executable bit loss

* fuzz: add util/mempool/h.cpp

Moving the mempool code (Boost) out of util.h, results in a ~10% speedup
(for me) when compiling the fuzz tests.

* fuzz: pass max fee into ConsumeTxMemPoolEntry

* refactor: Make 64-bit shift explicit

Also this change enables MSVC warning C4334 for all codebase.

https://docs.microsoft.com/en-us/cpp/error-messages/compiler-warnings/compiler-warning-level-3-c4334

* refactor: move run_command from util to common

Quoting ryanofsky: "util can be the library for things included in the kernel
which the kernel can depend on, and common can be the library for other code
that needs to be shared internally, but should not be part of the kernel or
shared externally."

* Remove clang-format from lint task

clang-format could be used in scripted diffs, but remained largely
unused.

* test: Pass mempool reference to AssemblerForTest

* test: Use dedicated mempool in TestPrioritisedMining

No need for a shared mempool. Also remove unused chainparams parameter.

* test: Use dedicated mempool in TestPackageSelection

No need for a shared mempool. Also remove unused chainparams parameter.

* test: Use dedicated mempool in TestBasicMining

No need for a shared mempool. Also remove unused chainparams parameter.

Can be reviewed with --ignore-all-space

* refactor: mempool: add MemPoolLimits::NoLimits()

There are quite a few places in the codebase that require us to
construct a CTxMemPool without limits on ancestors and descendants.
This helper function allows us to get rid of all that duplication.

* refactor: mempool: use CTxMempool::Limits

Simplifies function signatures by removing repetition of all the
ancestor/descendant limits,  and increases readability by being
more verbose by naming the limits, while still reducing the LoC.

* test: use NoLimits() in MempoolIndexingTest

The (100, 1000000, 1000, 1000000) limits are arbitrarily high and
don't restrict anything, they are just meant to calculate ancestors
properly. Using NoLimits() makes this intent more clear and simplifies
the code.

* docs: improve docs where MemPoolLimits is used

* Remove unused CDataStream::rdbuf method

It is unused and seems unlikely to be ever used.

* index: Improve BaseIndex::BlockUntilSyncedToCurrentChain reliability

Since commit f08c9fb from PR
bitcoin#21726, index
`BlockUntilSyncedToCurrentChain` behavior has been less reliable, and there has
also been a race condition in the `coinstatsindex_initial_sync` unit test.

It seems better for `BlockUntilSyncedToCurrentChain` to actually wait for the
last connected block to be fully processed, than to be able to return before
prune locks are set, so this switches the order of `m_best_block_index =
block;` and `UpdatePruneLock` statements in `SetBestBlockIndex` to make it more
reliable.

Also since commit f08c9fb, there has been a
race condition in the `coinstatsindex_initial_sync` test. Before that commit,
the atomic index best block pointer `m_best_block_index` was updated as the
last step of `BaseIndex::BlockConnected`, so `BlockUntilSyncedToCurrentChain`
could safely be used in tests to wait for the last `BlockConnected`
notification to be finished before stopping and destroying the index. But
after that commit, calling `BlockUntilSyncedToCurrentChain` is no longer
sufficient, and there is a race between the test shutdown code which destroys
the index object and the new code introduced in that commit calling
`AllowPrune()` and `GetName()` on the index object. Reproducibility
instructions for this are in
bitcoin#25365 (comment)

This commit fixes the `coinstatsindex_initial_sync` race condition, even though
it will require an additional change to silence TSAN false positives,
bitcoin#26188, after it is fixed. So this
partially addresses but does not resolve the bug reporting TSAN errors
bitcoin#25365.

There is no known race condition outside of test code currently, because the
bitcoind `Shutdown` function calls `FlushBackgroundCallbacks` not
`BlockUntilSyncedToCurrentChain` to safely shut down.

Co-authored-by: Vasil Dimov <[email protected]>
Co-authored-by: MacroFake <[email protected]>

* test: Prevent UB in `minisketch_tests.cpp`

* test: Remove confusing DUMMY_P2WPKH_SCRIPT

* docs: fix m_children to be a member of CTxMemPoolEntry

* wallet: have prune error take precedence over assumedvalid

From Russ Yanofsky:

"Agree with all of Marco's points here and think this should be updated

If havePrune and hasAssumedValidChain are both true, better to show
havePrune error message.  Assumed-valid error message is vague and not
very actionable.  Would suggest "Error loading wallet. Wallet requires
blocks to be downloaded, and software does not currently support loading
wallets while blocks are being downloaded out of order though assumeutxo
snapshots. Wallet should be able to load successfully after node sync
reaches height {block_height}"

Co-authored-by: MacroFake <[email protected]>
Co-authored-by: Russ Yanofsky <[email protected]>

* Fix comment typos in scriptpubkeyman.cpp, wallet.cpp, wallet.h

Fix comment typos:
sigature -> signature
ponter -> pointer
it's key -> its key

* sync: avoid confusing name overlap (Mutex)

Use `MutexType` instead of `Mutex` for the template parameter of
`UniqueLock` because there is already a class named `Mutex` and the
naming overlap is confusing. `MutexType` is used elsewhere in `sync.h`.

* sync: remove DebugLock alias template

Use `UniqueLock` directly. Type deduction works just fine from the first
argument to the constructor of `UniqueLock`, so there is no need to
repeat

```cpp
UniqueLock<typename std::remove_reference<typename std::remove_pointer<decltype(cs)>::type>::type>
```

five times in the `LOCK` macros. Just `UniqueLock` suffices.

* sync: rename AnnotatedMixin::UniqueLock to AnnotatedMixin::unique_lock

This avoids confusion with the global `UniqueLock` and the snake case
is consistent with `UniqueLock::reverse_lock.

* iwyu: Add zmq source files

* Update .cirrus.yml

Co-authored-by: brunoerg <[email protected]>
Co-authored-by: James O'Beirne <[email protected]>
Co-authored-by: Hennadii Stepanov <[email protected]>
Co-authored-by: Vasil Dimov <[email protected]>
Co-authored-by: MacroFake <[email protected]>
Co-authored-by: fanquake <[email protected]>
Co-authored-by: Anthony Towns <[email protected]>
Co-authored-by: willcl-ark <[email protected]>
Co-authored-by: Aurèle Oulès <[email protected]>
Co-authored-by: Sebastian Falbesoner <[email protected]>
Co-authored-by: glozow <[email protected]>
Co-authored-by: Cory Fields <[email protected]>
Co-authored-by: stickies-v <[email protected]>
Co-authored-by: Ryan Ofsky <[email protected]>
Co-authored-by: MacroFake <[email protected]>
Co-authored-by: Russ Yanofsky <[email protected]>
Co-authored-by: Dimitris Tsapakidis <[email protected]>
Co-authored-by: alex v <[email protected]>