Skip to content

Conversation

@JeremyRubin
Copy link
Contributor

@JeremyRubin JeremyRubin commented Sep 3, 2021

If multiple callers call ParseOpCode concurrently it will cause a race condition. We can either move the static to it's own area and require init be called explicitly, or just allow concurrent first callers to race to fill in an atomic variable that never changes thereafter. The second approach is taken here.

Static initialization is threadsafe, but only insofar as definining the variable being guaranteed to be called once. This is used incorrectly here.

practicalswift --> are there tools we can deploy to catch usage like this?

@ajtowns
Copy link
Contributor

ajtowns commented Sep 3, 2021

Wouldn't it be simpler to do something like:

class MapOpNames : public std::map<std::string, opcodetype>
{
public:
    MapOpNames()
    {
        for (unsigned int op = 0; op <= MAX_OPCODE; op++) {
            ...
        }
    }
};
opcodetype ParseOpCode(const std::string& s)
{
    const static MapOpNames mapOpNames;
    auto it = mapOpNames.find(s);
    ...
}

?

@kirillkovalenko
Copy link

maybe just move map initialization out of ParseOpCode?

static std::map<std::string, opcodetype> get_map()
{
    std::map<std::string, opcodetype> map;
    for (unsigned int op = 0; op <= MAX_OPCODE; op++) {
          ...    
    }
    return map;
};

opcodetype ParseOpCode(const std::string& s)
{
    const static std::map<std::string, opcodetype> mapOpNames(get_map());
    auto it = mapOpNames.find(s);
    ...
}

@JeremyRubin
Copy link
Contributor Author

Those are both solid suggestions, I guess I wasn't familiar enough with initializer rules to see those would be safe. Will probably go with @kirillkovalenko's approach

@JeremyRubin
Copy link
Contributor Author

Although thinking about it it's not clear to me that in your version get_map isn't called every time the function is run?

@maflcko
Copy link
Member

maflcko commented Sep 7, 2021

Why would it? You can test yourself:

#include <iostream>

static int get_map() {
  std::cout << __func__ << std::endl;
  return 42;
};

bool ParseOpCode() {
  const static auto mapOpNames{get_map()};
  return mapOpNames == 1;
}

int main() {
  ParseOpCode();
  ParseOpCode();
}

@kirillkovalenko
Copy link

Although thinking about it it's not clear to me that in your version get_map isn't called every time the function is run?

It's the way language/runtime implements it, basically a guarantee
https://en.cppreference.com/w/cpp/language/storage_duration#Static_local_variables

@JeremyRubin
Copy link
Contributor Author

JeremyRubin commented Sep 7, 2021 via email

@maflcko
Copy link
Member

maflcko commented Sep 8, 2021

There are issues in other parts of the codebase if this assumption doesn't hold

@JeremyRubin
Copy link
Contributor Author

reading the spec more closely I think it should be ok

@ajtowns
Copy link
Contributor

ajtowns commented Sep 9, 2021

I think you could also do:

namespace {
class OpCodeParser
{
private:
    std::map<std::string, opcodetype> mapOpNames;

public:
    OpCodeParser()
    {
        for (unsigned int op = 0; op <= MAX_OPCODE; op++) {
            ...
        }
    }
    opcodetype Parse(const std::string& s) const {
        auto it = mapOpNames.find(s);
        ...
    }
};    
} // namespace

opcodetype ParseOpCode(const std::string& s)
{
    const static OpCodeParser ocp;
    return ocp.Parse(s);
}

@JeremyRubin
Copy link
Contributor Author

went with @ajtowns suggested edit.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: according to clang-format:

diff --git a/src/core_read.cpp b/src/core_read.cpp
index bd29e2073e..ba535f745f 100644
--- a/src/core_read.cpp
+++ b/src/core_read.cpp
@@ -45,7 +45,8 @@ public:
             }
         }
     }
-    opcodetype Parse(const std::string& s) const {
+    opcodetype Parse(const std::string& s) const
+    {
         auto it = mapOpNames.find(s);
         if (it == mapOpNames.end()) throw std::runtime_error("script parse error: unknown opcode");
         return it->second;

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit coding-style: opening brace for functions/classes on a separate line.

@DrahtBot
Copy link
Contributor

DrahtBot commented Oct 8, 2021

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

No conflicts as of last run.

luke-jr pushed a commit to bitcoinknots/bitcoin that referenced this pull request Oct 10, 2021
Github-Pull: bitcoin#22875
Rebased-From: d5e006c84a12b194a27e4eb0c620da8fbc113c18
@maflcko
Copy link
Member

maflcko commented Oct 22, 2021

Are you still working on this? If not, I suggest to close this PR

@maflcko
Copy link
Member

maflcko commented Oct 22, 2021

practicalswift --> are there tools we can deploy to catch usage like this?

bitcoin-tx is single threaded. I don't think tooling exists to catch "races" in single-threaded programs, whatever that means.

@JeremyRubin
Copy link
Contributor Author

well i think core_read is still included in libbitcoin sources so i think it should be threadsafe -- good rule of thumb is that any pure function should be threadsafe, always.

i don't think closing the PR is a good idea; i'll try to rebase it when i have time.

@laanwj
Copy link
Member

laanwj commented Nov 26, 2021

good rule of thumb is that any pure function should be threadsafe, always.

There's some risk that this function gets moved to an utility at some point when it is needed somewhere else, and people forget to make it thread-safe.

I'm fine with making this change, but please address the comments.

luke-jr pushed a commit to bitcoinknots/bitcoin that referenced this pull request Dec 14, 2021
Github-Pull: bitcoin#22875
Rebased-From: d5e006c84a12b194a27e4eb0c620da8fbc113c18
@JeremyRubin
Copy link
Contributor Author

rebased, nits addressed

Copy link
Member

@sipa sipa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review ACK cfa2b8dbf9fc6d020281e6521e83853838f62f81

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit coding-style: opening brace for functions/classes on a separate line.

@maflcko
Copy link
Member

maflcko commented Dec 22, 2021

review ACK cfa2b8dbf9fc6d020281e6521e83853838f62f81 🗡

Show signature

Signature:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

review ACK cfa2b8dbf9fc6d020281e6521e83853838f62f81 🗡
-----BEGIN PGP SIGNATURE-----

iQGzBAEBCgAdFiEE+rVPoUahrI9sLGYTzit1aX5ppUgFAlwqrYAACgkQzit1aX5p
pUie4gv/ZLgNcH13qKel1nfbVmVAe4FRMQ5/D76URcNUf6jNb0xbgcSloTRhCx+j
d4aFKjPrCpo2qzpRO3i7llmM3HND48EQuMM2he2dKqZVCfh2q5I8YZcAokBPpsw3
5g/4LrPaKTcaINwD42txaJX6yNG0oTuM2xabTatfeAq310GMSw0mDqKCBNkw+9vO
N34Un7/mWR+zv1EQLqGI2SsJiYoI7MQjh8dH9e35gE19Z+fsthebEsccSy+EcbYq
ZGQZiANnmvGDUzExjg0/6isxmwf8uJBfDFztKPuSV+FY0jG2oT9scS8B8p9H1q88
zFxt02MJfUegj0i6O3UKp4L5qJnvICEIdfDIUyKBTENYj/hLiw5gbjPvJABhn+lW
Ef6NArWBrSPGrb15pGk8CjwefZIAW+vgNyQ/EUR3ihPnXssO/8i4Fe75ic9Ib6Xy
7p/l/ZhH8noG1J8zFMe5OnLKeFAmZNMg3q+zenHkacmI9V24ttV1+xwWmuWmf4wb
+yXKFOKH
=Q+n8
-----END PGP SIGNATURE-----

@maflcko
Copy link
Member

maflcko commented Dec 22, 2021

@JeremyRubin would be really nice to fix the two nits, but this can be also be merged as-is. Please let us know what you prefer.

@JeremyRubin
Copy link
Contributor Author

since the current PR has 2 code review ACKs on cfa2b8d, i don't want to burden anyone with re-review on that hash, so I pushed a fixup with it. If you want i can squash it, or you can on merge.

@maflcko
Copy link
Member

maflcko commented Dec 23, 2021

@maflcko
Copy link
Member

maflcko commented Dec 23, 2021

maintainers aren't allowed to squash, as it would break the signature check script

@maflcko maflcko changed the title Fix Racy ParseOpCode function initialization util: Fix Racy ParseOpCode function initialization Dec 23, 2021
@JeremyRubin
Copy link
Contributor Author

squashed

Copy link
Member

@maflcko maflcko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

re-ACK 7b481f0 🗣

Show signature

Signature:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

re-ACK 7b481f015a0386f5ee3e13415474952653ddc198 🗣
-----BEGIN PGP SIGNATURE-----

iQGzBAEBCgAdFiEE+rVPoUahrI9sLGYTzit1aX5ppUgFAlwqrYAACgkQzit1aX5p
pUgMsAv+IixBito8qp9aWseDFUHP4taa+OQ4iO4dEf7Qt8MvAUY/vLXATBx4cuUJ
5ZORW+54b4ZYCV4Qw9I7FmxKZBY9ihREr7/JjvteAtJSVULv35s6zxJxESnec80q
76ynV3KYnKd6pNMCHmR1Gy2EyIAUtG1ttGHkE5TfkjfL/WVEBBC0Yziwc0Zc1Tr9
gcvyLBWP9h84Lo1KS6ul8NTAJvk7e1r401S9DsF+1mqMrMrLUCY/2l19VqUBV9J0
jsPJOO/LHcJbTt2biXx16NtVwCDbr590fZPSIvFJ7kmb4p0M3Fw0BI+EgucTSSKM
+JOTf+v31zuacSRjpgqmjKDNjb0rodKavcJEJG1IOwakCs4eZdwAx1W1y4Bh4672
K/Zhsb9cQl1VAVJrFLLNlinsi6kqz6490MfxKg7COHXz4YNaUuqpiulQo/zbbRW6
L1EhPhuKs42ZvHkRecOzi+wEC+8XvBIuQ+Bx5kD0oH530m697kBdnRgbAlKVB9yX
rr9jAuD7
=bTsy
-----END PGP SIGNATURE-----

@maflcko maflcko merged commit dada92f into bitcoin:master Dec 24, 2021
sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Dec 27, 2021
7b481f0 Fix Racy ParseOpCode function initialization (Jeremy Rubin)

Pull request description:

  If multiple callers call ParseOpCode concurrently it will cause a race condition. We can either move the static to it's own area and require init be called explicitly, or just allow concurrent first callers to race to fill in an atomic variable that never changes thereafter. The second approach is taken here.

  Static initialization *is* threadsafe, but only insofar as definining the variable being guaranteed to be called once. This is used incorrectly here.

  practicalswift --> are there tools we can deploy to catch usage like this?

ACKs for top commit:
  MarcoFalke:
    re-ACK 7b481f0 🗣

Tree-SHA512: cbf9dc3af26d7335305026f32ce8472a018309b89b3d81a67357e59fbeed72c37b5b8a6e30325ea68145c3b2403867be82de01f22decefb6e6717cf0c0045633
@bitcoin bitcoin locked and limited conversation to collaborators Dec 24, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants