Skip to content

Conversation

@theStack
Copy link
Contributor

@theStack theStack commented Jan 23, 2021

It seems more natural to treat the "subversion" field (=user agent string, see BIP 14) of a node as pure string rather than a bytestring within the test framework. This is also suggested with the naming prefix in msg_version.strSubVer: one probably wouldn't expect a field starting with "str" to be a bytestring that needs further decoding to be useful. This PR moves the encoding/decoding parts to the serialization/deserialization routines so that the user doesn't have to bother with that anymore.

Note that currently, in the master branch the msg_version.strSubVer is never read (only in msg_version.__repr__); However, one issue that is solved by this PR came up while testing #19509 (not merged yet): A decoding script for binary message capture files takes use of the functional test framework convert it into JSON format. Bytestrings will be convered to hexstrings, while pure strings will (surprise surprise) end up without modification in the file.

So without this patch, we get:

$ jq . out.json | grep -m5 strSubVer
      "strSubVer": "2f5361746f7368693a32312e39392e302f"
      "strSubVer": "2f5361746f7368693a302e32302e312f"
      "strSubVer": "2f5361746f7368693a32312e39392e302f"
      "strSubVer": "2f5361746f7368693a302e32302e312f"
      "strSubVer": "2f5361746f7368693a32312e39392e302f"

After this patch:

$ jq . out2.json | grep -m5 strSubVer
      "strSubVer": "/Satoshi:21.99.0/"
      "strSubVer": "/Satoshi:0.20.1/"
      "strSubVer": "/Satoshi:21.99.0/"
      "strSubVer": "/Satoshi:0.20.1/"
      "strSubVer": "/Satoshi:21.99.0/"

@DrahtBot DrahtBot added the Tests label Jan 23, 2021
@laanwj
Copy link
Member

laanwj commented Jan 23, 2021

Mind that subversion strings on the P2P network are not guaranteed to be valid UTF-8 encoded. That said, for the test framework this assumption is fine.

@DrahtBot
Copy link
Contributor

DrahtBot commented Jan 23, 2021

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@jnewbery
Copy link
Contributor

utACK de85af5

This change is part of #20524, but it seems fine to make the change separate from that PR. This fixes potential silent failures like the one seen in #20522.

r += self.addrFrom.serialize(with_time=False)
r += struct.pack("<Q", self.nNonce)
r += ser_string(self.strSubVer)
r += ser_string(self.strSubVer.encode('utf-8'))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bitcoin Core will only produce and read ascii

Suggested change
r += ser_string(self.strSubVer.encode('utf-8'))
r += ser_string(self.strSubVer.encode('ascii'))

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only in the test framework. Outside it, the user could provide a uacomment in bitcoin.conf with some emoji in it. Or, even non-UTF-8. There, IIRC, are no checks on the content.
(sorry for resolving this, apparently github thought that should be the function of my space button)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure? SAFE_CHARS_UA_COMMENT only contains the ranges a-z0-9 and the chars .,;-_?@. See commit 1c1b1b3

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought that was only for printing/logging/RPC etc, but seems you're right.

@maflcko maflcko merged commit 69f7f50 into bitcoin:master Feb 17, 2021
sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Feb 17, 2021
Fabcien pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this pull request Jan 31, 2022
Summary:
> It seems more natural to treat the "subversion" field (=user agent string, see BIP 14) of a node as pure string rather than a bytestring within the test framework. This is also suggested with the naming prefix in msg_version.strSubVer: one probably wouldn't expect a field starting with "str" to be a bytestring that needs further decoding to be useful. This PR moves the encoding/decoding parts to the serialization/deserialization routines so that the user doesn't have to bother with that anymore.

This is a backport of [[bitcoin/bitcoin#20993 | core#20993]]

Test Plan: `ninja check-functional-extended`

Reviewers: #bitcoin_abc, Fabien

Reviewed By: #bitcoin_abc, Fabien

Differential Revision: https://reviews.bitcoinabc.org/D10943
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Aug 16, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants