Skip to content

Commit 9b22095

Browse files
UdjinM6PastaPastaPasta
authored andcommitted
Various suggestions
Major Improvements ✅ 1. Simplified FindNode API // REMOVED the complex FindNodeLocked/FindNodeLockedMutable distinction - const CNode* FindNodeLocked(...) const SHARED_LOCKS_REQUIRED(m_nodes_mutex); - CNode* FindNodeLockedMutable(...) EXCLUSIVE_LOCKS_REQUIRED(m_nodes_mutex); // REPLACED with simple template-based approach template<typename Key> const CNode* FindNode(const Key& key, ...) const SHARED_LOCKS_REQUIRED(m_nodes_mutex); template<typename Key> CNode* FindNodeMutable(const Key& key, ...) SHARED_LOCKS_REQUIRED(m_nodes_mutex); Analysis: ✅ Much better! - Generic template handles all key types (CNetAddr, CService, NodeId, string) - Both require only SHARED_LOCKS_REQUIRED - correctly reflects that finding in the vector only needs shared lock - Mutable vs const is about pointer type, not lock type - Eliminates ~80 lines of duplicate code 2. Fixed getPendingProbes Lock Annotation // BEFORE: Incorrectly required m_nodes_mutex - const auto getPendingProbes = [&]() EXCLUSIVE_LOCKS_REQUIRED(m_nodes_mutex, cs_vPendingMasternodes) { - AssertLockHeld(m_nodes_mutex); // AFTER: Only requires cs_vPendingMasternodes (correct!) + const auto getPendingProbes = [&]() EXCLUSIVE_LOCKS_REQUIRED(cs_vPendingMasternodes) { Analysis: ✅ Correct! This fixes the issue we identified - getPendingProbes doesn't access m_nodes. 3. Fixed getPendingQuorumNodes Lock Annotation // BEFORE: Required exclusive lock - const auto getPendingQuorumNodes = [&]() EXCLUSIVE_LOCKS_REQUIRED(m_nodes_mutex, cs_vPendingMasternodes) { - AssertLockHeld(m_nodes_mutex); // AFTER: Only requires shared lock (correct!) + const auto getPendingQuorumNodes = [&]() SHARED_LOCKS_REQUIRED(m_nodes_mutex) + EXCLUSIVE_LOCKS_REQUIRED(cs_vPendingMasternodes) { + AssertSharedLockHeld(m_nodes_mutex); Analysis: ✅ Correct! - Now properly uses SHARED_LOCKS_REQUIRED since it only reads from m_nodes - Uses AssertSharedLockHeld instead of AssertLockHeld - Writes to pnode->fDisconnect are safe (atomic) 4. Simplified getConnectToDmn Locking // BEFORE: Held exclusive lock - LOCK2(m_nodes_mutex, cs_vPendingMasternodes); // AFTER: Only holds shared lock for m_nodes + READ_LOCK(m_nodes_mutex); + LOCK(cs_vPendingMasternodes); Analysis: ✅ Excellent optimization! - Only needs shared lock since it's just reading from m_nodes - Reduces contention significantly 5. Improved Logic in getPendingQuorumNodes // BEFORE: Complex TOCTOU pattern - if (const CNode* pnode = FindNodeLocked(addr2)) { - slowHandshake = pnode->nTimeFirstMessageReceived.load() != 0s && ...; - } - if (slowHandshake) { - if (CNode* p2 = FindNodeLockedMutable(addr2)) { p2->fDisconnect = true; } - } // AFTER: Simplified single lookup + CNode* pnode = FindNodeMutable(addr2); + if (pnode && pnode->m_masternode_connection) { + continue; + } + if (connectedNodes.count(addr2)) { + bool slow_handshake = pnode && pnode->nTimeFirstMessageReceived.load() != 0s && ...; + if (slow_handshake) { + pnode->fDisconnect = true; // Direct access, no second lookup + } Analysis: ✅ Much cleaner! - Eliminates TOCTOU issue by using single pointer - More readable logic flow - Early connectedProRegTxHashes check avoids unnecessary work 6. Renamed Helper Methods for Clarity // BEFORE: Confusing names - WithNodeShared() - WithNodeExclusive() - ExistsNodeShared() // AFTER: Clear intent + WithNodeMutable() // Callback gets mutable CNode* + ExistsNode() // Simple existence check Analysis: ✅ Better naming! - WithNodeMutable clearly indicates you get a mutable pointer - No need for "Shared" suffix - the shared lock is an implementation detail - The lock type is in the function signature for type safety 7. Proper AssertSharedLockHeld Implementation // BEFORE: Hacky workaround with NO_THREAD_SAFETY_ANALYSIS - inline void AssertSharedLockHeldInline(...) NO_THREAD_SAFETY_ANALYSIS { - AssertLockHeldInternal(...); - } // AFTER: Proper template specialization + template <typename MutexType> + void AssertSharedLockHeldInternal(...) SHARED_LOCKS_REQUIRED(cs) { ... } + template void AssertSharedLockHeldInternal(..., SharedMutex*); Analysis: ✅ Much better! - Proper template implementation - Correct thread safety annotations - No need for NO_THREAD_SAFETY_ANALYSIS hack Remaining Observations ⚠️ FindNodeMutable with Shared Lock template<typename Key> CNode* FindNodeMutable(const Key& key, ...) SHARED_LOCKS_REQUIRED(m_nodes_mutex) Question: This returns a mutable CNode* under shared lock. Is this intentional? Answer: ✅ Yes, this is correct! - The shared lock protects the m_nodes container, not the CNode objects - CNode has internal thread safety - Returning mutable pointer allows modifying atomic members like fDisconnect - The "Mutable" in the name documents this is intentional ✅ IsMasternodeOrDisconnectRequested Now Const - bool IsMasternodeOrDisconnectRequested(const CService& addr); + bool IsMasternodeOrDisconnectRequested(const CService& addr) const; Analysis: ✅ Correct - it's a read-only operation. Summary of Improvements | Issue | Status | Notes | |---------------------------------------|---------|-----------------------------------| | Unnecessary FindNodeLocked variants | ✅ Fixed | Replaced with clean templates | | getPendingProbes incorrect lock | ✅ Fixed | Removed m_nodes_mutex requirement | | getPendingQuorumNodes wrong lock type | ✅ Fixed | Changed to SHARED_LOCKS_REQUIRED | | TOCTOU in masternode connection | ✅ Fixed | Single pointer lookup | | AssertSharedLockHeld hack | ✅ Fixed | Proper template implementation | | Confusing method names | ✅ Fixed | Better naming convention | | Over-locking in getConnectToDmn | ✅ Fixed | Changed to READ_LOCK | Reviewed-by: Claude (Anthropic AI Assistant)
1 parent c1b3047 commit 9b22095

File tree

4 files changed

+82
-175
lines changed

4 files changed

+82
-175
lines changed

src/net.cpp

Lines changed: 50 additions & 145 deletions
Original file line numberDiff line numberDiff line change
@@ -338,88 +338,10 @@ bool IsLocal(const CService& addr)
338338
return mapLocalHost.count(addr) > 0;
339339
}
340340

341-
// Internal FindNodeLocked - const versions do the actual work
342-
const CNode* CConnman::FindNodeLocked(const CNetAddr& ip, bool fExcludeDisconnecting) const
343-
{
344-
for (const CNode* pnode : m_nodes) {
345-
if (fExcludeDisconnecting && pnode->fDisconnect) {
346-
continue;
347-
}
348-
if (static_cast<CNetAddr>(pnode->addr) == ip) {
349-
return pnode;
350-
}
351-
}
352-
return nullptr;
353-
}
354-
355-
const CNode* CConnman::FindNodeLocked(const std::string& addrName, bool fExcludeDisconnecting) const
356-
{
357-
for (const CNode* pnode : m_nodes) {
358-
if (fExcludeDisconnecting && pnode->fDisconnect) {
359-
continue;
360-
}
361-
if (pnode->m_addr_name == addrName) {
362-
return pnode;
363-
}
364-
}
365-
return nullptr;
366-
}
367-
368-
const CNode* CConnman::FindNodeLocked(const CService& addr, bool fExcludeDisconnecting) const
369-
{
370-
for (const CNode* pnode : m_nodes) {
371-
if (fExcludeDisconnecting && pnode->fDisconnect) {
372-
continue;
373-
}
374-
if (static_cast<CService>(pnode->addr) == addr) {
375-
return pnode;
376-
}
377-
}
378-
return nullptr;
379-
}
380-
381-
const CNode* CConnman::FindNodeLocked(NodeId id, bool fExcludeDisconnecting) const
382-
{
383-
for (const CNode* pnode : m_nodes) {
384-
if (fExcludeDisconnecting && pnode->fDisconnect) {
385-
continue;
386-
}
387-
if (pnode->GetId() == id) {
388-
return pnode;
389-
}
390-
}
391-
return nullptr;
392-
}
393-
394-
CNode* CConnman::FindNodeLockedMutable(const CNetAddr& ip, bool fExcludeDisconnecting)
395-
{
396-
AssertLockHeld(m_nodes_mutex);
397-
return const_cast<CNode*>(FindNodeLocked(ip, fExcludeDisconnecting));
398-
}
399-
400-
CNode* CConnman::FindNodeLockedMutable(const std::string& addrName, bool fExcludeDisconnecting)
401-
{
402-
AssertLockHeld(m_nodes_mutex);
403-
return const_cast<CNode*>(FindNodeLocked(addrName, fExcludeDisconnecting));
404-
}
405-
406-
CNode* CConnman::FindNodeLockedMutable(const CService& addr, bool fExcludeDisconnecting)
407-
{
408-
AssertLockHeld(m_nodes_mutex);
409-
return const_cast<CNode*>(FindNodeLocked(addr, fExcludeDisconnecting));
410-
}
411-
412-
CNode* CConnman::FindNodeLockedMutable(NodeId id, bool fExcludeDisconnecting)
413-
{
414-
AssertLockHeld(m_nodes_mutex);
415-
return const_cast<CNode*>(FindNodeLocked(id, fExcludeDisconnecting));
416-
}
417-
418-
419341
bool CConnman::AlreadyConnectedToAddress(const CAddress& addr) const
420342
{
421343
READ_LOCK(m_nodes_mutex);
422-
return FindNodeLocked(static_cast<CService>(addr)) != nullptr;
344+
return FindNode(static_cast<CService>(addr)) != nullptr;
423345
}
424346

425347
bool CConnman::CheckIncomingNonce(uint64_t nonce) const
@@ -457,8 +379,7 @@ CNode* CConnman::ConnectNode(CAddress addrConnect, const char *pszDest, bool fCo
457379
}
458380

459381
// Look for an existing connection
460-
if (WithNodeShared(static_cast<CService>(addrConnect), [](const CNode*){ return true; }))
461-
{
382+
if (ExistsNode(static_cast<CService>(addrConnect))) {
462383
LogPrintf("Failed to open new connection, already connected\n");
463384
return nullptr;
464385
}
@@ -493,7 +414,7 @@ CNode* CConnman::ConnectNode(CAddress addrConnect, const char *pszDest, bool fCo
493414
}
494415
// It is possible that we already have a connection to the IP/port pszDest resolved to.
495416
// In that case, drop the connection that was just created.
496-
if (ExistsNodeShared(static_cast<CService>(addrConnect))) {
417+
if (ExistsNode(static_cast<CService>(addrConnect))) {
497418
LogPrintf("Not opening a connection to %s, already connected to %s\n", pszDest, addrConnect.ToStringAddrPort());
498419
return nullptr;
499420
}
@@ -1882,7 +1803,7 @@ bool CConnman::AttemptToEvictConnection()
18821803
if (!node_id_to_evict) {
18831804
return false;
18841805
}
1885-
return WithNodeExclusive(*node_id_to_evict, [](CNode* pnode){
1806+
return WithNodeMutable(*node_id_to_evict, [](CNode* pnode){
18861807
LogPrint(BCLog::NET_NETCONN, "selected %s connection for eviction peer=%d; disconnecting\n", pnode->ConnectionTypeAsString(), pnode->GetId());
18871808
pnode->fDisconnect = true;
18881809
return true;
@@ -3447,52 +3368,53 @@ void CConnman::ThreadOpenMasternodeConnections(CDeterministicMNManager& dmnman,
34473368

34483369
MasternodeProbeConn isProbe = MasternodeProbeConn::IsNotConnection;
34493370

3450-
const auto getPendingQuorumNodes = [&]() EXCLUSIVE_LOCKS_REQUIRED(m_nodes_mutex, cs_vPendingMasternodes) {
3451-
AssertLockHeld(m_nodes_mutex);
3371+
const auto getPendingQuorumNodes = [&]() SHARED_LOCKS_REQUIRED(m_nodes_mutex) EXCLUSIVE_LOCKS_REQUIRED(cs_vPendingMasternodes) {
3372+
AssertSharedLockHeld(m_nodes_mutex);
34523373
AssertLockHeld(cs_vPendingMasternodes);
34533374
std::vector<CDeterministicMNCPtr> ret;
34543375
for (const auto& group : masternodeQuorumNodes) {
34553376
for (const auto& proRegTxHash : group.second) {
3377+
if (connectedProRegTxHashes.count(proRegTxHash)) {
3378+
continue;
3379+
}
34563380
auto dmn = mnList.GetMN(proRegTxHash);
34573381
if (!dmn) {
34583382
continue;
34593383
}
34603384
const auto addr2 = dmn->pdmnState->netInfo->GetPrimary();
3461-
if (connectedNodes.count(addr2) && !connectedProRegTxHashes.count(proRegTxHash)) {
3385+
CNode* pnode = FindNodeMutable(addr2);
3386+
if (pnode && pnode->m_masternode_connection) {
3387+
// node is masternode, skip it
3388+
continue;
3389+
}
3390+
if (connectedNodes.count(addr2)) {
34623391
// we probably connected to it before it became a masternode
34633392
// or maybe we are still waiting for mnauth
3464-
// Use shared access since we already hold m_nodes_mutex
3465-
bool slowHandshake = false;
3466-
if (const CNode* pnode = FindNodeLocked(addr2)) {
3467-
slowHandshake = pnode->nTimeFirstMessageReceived.load() != 0s && GetTime<std::chrono::seconds>() - pnode->nTimeFirstMessageReceived.load() > 5s;
3468-
}
3469-
if (slowHandshake) {
3393+
bool slow_handshake = pnode && pnode->nTimeFirstMessageReceived.load() != 0s &&
3394+
GetTime<std::chrono::seconds>() - pnode->nTimeFirstMessageReceived.load() > 5s;
3395+
if (slow_handshake) {
34703396
// clearly not expecting mnauth to take that long even if it wasn't the first message
34713397
// we received (as it should normally), disconnect
3472-
LogPrint(BCLog::NET_NETCONN, "CConnman::%s -- dropping non-mnauth connection to %s, service=%s\n", _func_, proRegTxHash.ToString(), addr2.ToStringAddrPort());
3473-
if (CNode* p2 = FindNodeLockedMutable(addr2)) { p2->fDisconnect = true; }
3398+
LogPrint(BCLog::NET_NETCONN, "CConnman::%s -- dropping non-mnauth connection to %s, service=%s\n",
3399+
_func_, proRegTxHash.ToString(), addr2.ToStringAddrPort());
3400+
pnode->fDisconnect = true;
34743401
}
34753402
// either way - it's not ready, skip it for now
34763403
continue;
34773404
}
3478-
// Check if node is masternode or disconnect requested using FindNodeLocked
3479-
const CNode* existingNode = FindNodeLocked(addr2);
3480-
bool isDisconnectRequested = existingNode && (existingNode->m_masternode_connection || existingNode->fDisconnect);
3481-
if (!connectedNodes.count(addr2) && !isDisconnectRequested && !connectedProRegTxHashes.count(proRegTxHash)) {
3482-
int64_t lastAttempt = mn_metaman.GetMetaInfo(dmn->proTxHash)->GetLastOutboundAttempt();
3483-
// back off trying connecting to an address if we already tried recently
3484-
if (nANow - lastAttempt < chainParams.LLMQConnectionRetryTimeout()) {
3485-
continue;
3486-
}
3487-
ret.emplace_back(dmn);
3405+
// back off connecting to an address if we already tried recently
3406+
int64_t last_attempt = mn_metaman.GetMetaInfo(dmn->proTxHash)->GetLastOutboundAttempt();
3407+
if (nANow - last_attempt < chainParams.LLMQConnectionRetryTimeout()) {
3408+
continue;
34883409
}
3410+
// all checks passed
3411+
ret.emplace_back(dmn);
34893412
}
34903413
}
34913414
return ret;
34923415
};
34933416

3494-
const auto getPendingProbes = [&]() EXCLUSIVE_LOCKS_REQUIRED(m_nodes_mutex, cs_vPendingMasternodes) {
3495-
AssertLockHeld(m_nodes_mutex);
3417+
const auto getPendingProbes = [&]() EXCLUSIVE_LOCKS_REQUIRED(cs_vPendingMasternodes) {
34963418
AssertLockHeld(cs_vPendingMasternodes);
34973419
std::vector<CDeterministicMNCPtr> ret;
34983420
for (auto it = masternodePendingProbes.begin(); it != masternodePendingProbes.end(); ) {
@@ -3523,24 +3445,23 @@ void CConnman::ThreadOpenMasternodeConnections(CDeterministicMNManager& dmnman,
35233445

35243446
auto getConnectToDmn = [&]() -> CDeterministicMNCPtr {
35253447
// don't hold lock while calling OpenMasternodeConnection as cs_main is locked deep inside
3526-
LOCK2(m_nodes_mutex, cs_vPendingMasternodes);
3448+
READ_LOCK(m_nodes_mutex);
3449+
LOCK(cs_vPendingMasternodes);
35273450

35283451
if (!vPendingMasternodes.empty()) {
35293452
auto dmn = mnList.GetValidMN(vPendingMasternodes.front());
35303453
vPendingMasternodes.erase(vPendingMasternodes.begin());
35313454
// Check if we should connect to this masternode
3532-
// We already hold m_nodes_mutex here, so check fDisconnect and m_masternode_connection directly
3533-
bool shouldConnect = true;
3455+
// We already hold m_nodes_mutex here, so check m_masternode_connection directly
35343456
if (dmn && !connectedNodes.count(dmn->pdmnState->netInfo->GetPrimary())) {
3535-
bool anyBad = false;
3536-
if (const CNode* pnode = FindNodeLocked(dmn->pdmnState->netInfo->GetPrimary())) {
3537-
anyBad = pnode->m_masternode_connection || pnode->fDisconnect;
3457+
if (const CNode* pnode = FindNode(dmn->pdmnState->netInfo->GetPrimary())) {
3458+
if (!pnode->m_masternode_connection) {
3459+
LogPrint(BCLog::NET_NETCONN, "CConnman::%s -- opening pending masternode connection to %s, service=%s\n",
3460+
_func_, dmn->proTxHash.ToString(),
3461+
dmn->pdmnState->netInfo->GetPrimary().ToStringAddrPort());
3462+
return dmn;
3463+
}
35383464
}
3539-
shouldConnect = !anyBad;
3540-
}
3541-
if (dmn && shouldConnect) {
3542-
LogPrint(BCLog::NET_NETCONN, "CConnman::%s -- opening pending masternode connection to %s, service=%s\n", _func_, dmn->proTxHash.ToString(), dmn->pdmnState->netInfo->GetPrimary().ToStringAddrPort());
3543-
return dmn;
35443465
}
35453466
}
35463467

@@ -3626,8 +3547,9 @@ void CConnman::OpenNetworkConnection(const CAddress& addrConnect, bool fCountFai
36263547
if (addrConnect.GetPort() == GetListenPort() && IsLocal(addrConnect)) {
36273548
return;
36283549
}
3629-
} else if (WithNodeShared(std::string(pszDest), [](const CNode*){ return true; }))
3550+
} else if (ExistsNode(std::string(pszDest))) {
36303551
return;
3552+
}
36313553

36323554
LogPrint(BCLog::NET_NETCONN, "CConnman::%s -- connecting to %s\n", __func__, getIpStr());
36333555
CNode* pnode = ConnectNode(addrConnect, pszDest, fCountFailure, conn_type, use_v2transport);
@@ -4533,7 +4455,7 @@ void CConnman::GetNodeStats(std::vector<CNodeStats>& vstats) const
45334455

45344456
bool CConnman::DisconnectNode(const std::string& strNode)
45354457
{
4536-
return WithNodeExclusive(strNode, [&](CNode* pnode){
4458+
return WithNodeMutable(strNode, [&](CNode* pnode){
45374459
LogPrint(BCLog::NET_NETCONN, "disconnect by address%s matched peer=%d; disconnecting\n", (fLogIPs ? strprintf("=%s", strNode) : ""), pnode->GetId());
45384460
pnode->fDisconnect = true;
45394461
return true;
@@ -4561,7 +4483,7 @@ bool CConnman::DisconnectNode(const CNetAddr& addr)
45614483

45624484
bool CConnman::DisconnectNode(NodeId id)
45634485
{
4564-
return WithNodeExclusive(id, [&](CNode* pnode){
4486+
return WithNodeMutable(id, [&](CNode* pnode){
45654487
LogPrint(BCLog::NET_NETCONN, "disconnect by id peer=%d; disconnecting\n", pnode->GetId());
45664488
pnode->fDisconnect = true;
45674489
return true;
@@ -4816,50 +4738,33 @@ void CConnman::PushMessage(CNode* pnode, CSerializedNetMsg&& msg)
48164738
bool CConnman::ForNode(const CService& addr, std::function<bool(const CNode* pnode)> cond, std::function<bool(CNode* pnode)> func)
48174739
{
48184740
READ_LOCK(m_nodes_mutex);
4819-
CNode* found = nullptr;
4820-
for (auto&& pnode : m_nodes) {
4821-
if((CService)pnode->addr == addr) {
4822-
found = pnode;
4823-
break;
4824-
}
4825-
}
4741+
CNode* found = FindNodeMutable(addr, false);
48264742
return found != nullptr && cond(found) && func(found);
48274743
}
48284744

48294745
bool CConnman::ForNode(const CService& addr, std::function<bool(const CNode* pnode)> cond, std::function<bool(const CNode* pnode)> func) const
48304746
{
48314747
READ_LOCK(m_nodes_mutex);
4832-
const CNode* found = FindNodeLockedBy(addr, false);
4748+
const CNode* found = FindNode(addr, false);
48334749
return found != nullptr && cond(found) && func(found);
48344750
}
48354751

4836-
bool CConnman::ForNode(NodeId id, std::function<bool(const CNode* pnode)> cond, std::function<bool(const CNode* pnode)> func) const
4752+
bool CConnman::ForNode(NodeId id, std::function<bool(const CNode* pnode)> cond, std::function<bool(CNode* pnode)> func)
48374753
{
4838-
CNode* found = nullptr;
48394754
READ_LOCK(m_nodes_mutex);
4840-
for (auto&& pnode : m_nodes) {
4841-
if(pnode->GetId() == id) {
4842-
found = pnode;
4843-
break;
4844-
}
4845-
}
4755+
CNode* found = FindNodeMutable(id, false);
48464756
return found != nullptr && cond(found) && func(found);
48474757
}
48484758

4849-
bool CConnman::ForNode(NodeId id, std::function<bool(const CNode* pnode)> cond, std::function<bool(CNode* pnode)> func)
4759+
bool CConnman::ForNode(NodeId id, std::function<bool(const CNode* pnode)> cond, std::function<bool(const CNode* pnode)> func) const
48504760
{
4851-
CNode* found = nullptr;
48524761
READ_LOCK(m_nodes_mutex);
4853-
for (auto&& pnode : m_nodes) {
4854-
if(pnode->GetId() == id) {
4855-
found = pnode;
4856-
break;
4857-
}
4858-
}
4762+
const CNode* found = FindNode(id, false);
48594763
return found != nullptr && cond(found) && func(found);
48604764
}
48614765

4862-
bool CConnman::IsMasternodeOrDisconnectRequested(const CService& addr) {
4766+
bool CConnman::IsMasternodeOrDisconnectRequested(const CService& addr) const
4767+
{
48634768
return ForNode(addr, AllNodes, [](const CNode* pnode){
48644769
return pnode->m_masternode_connection || pnode->fDisconnect;
48654770
});

0 commit comments

Comments
 (0)