Skip to content

Conversation

@theStack
Copy link
Contributor

This PR simplifies the linearization scripts linearize-data.py and linearize-hashes.py by replacing overly complicated cruft (block header hash string calculation, hex string reverse) with means of the Python3 standard library.

…data.py

The string representation of a block header hash is simply the
hexlified byte-reversed double SHA256 hash of its serialization.
Switching the endianness of a hex string `str` can simply be
achieved by `bytes.fromhex(str)[::-1].hex()`, i.e. we can use
that and remove those helper methods.
@laanwj
Copy link
Member

laanwj commented Feb 22, 2022

Concept ACK
NIce cleanup!

@laanwj
Copy link
Member

laanwj commented Feb 23, 2022

It reminds me that we don't have any kind of test for these scripts. It might be good to have a standardized test, even if it involves manual steps. If anyone is still using these scripts at all? At the time linearize was introduced it was used to create a blockchain file for distribution through torrent. I don't think anyone has done that for a long time. But maybe it has other users.

@theStack
Copy link
Contributor Author

It reminds me that we don't have any kind of test for these scripts. It might be good to have a standardized test, even if it involves manual steps. If anyone is still using these scripts at all? At the time linearize was introduced it was used to create a blockchain file for distribution through torrent. I don't think anyone has done that for a long time. But maybe it has other users.

Currently, the functional test feature_loadblock.py uses it to generate a serialized blockchain (bootstrap.dat), which is then loaded via -loadblock:

self.log.info("Run linearization of block hashes")
linearize_hashes_file = os.path.join(linearize_dir, "linearize-hashes.py")
subprocess.run([sys.executable, linearize_hashes_file, cfg_file],
stdout=hash_list,
check=True)
self.log.info("Run linearization of block data")
linearize_data_file = os.path.join(linearize_dir, "linearize-data.py")
subprocess.run([sys.executable, linearize_data_file, cfg_file],
check=True)

I agree though that having more detailled tests would probably be nice.

@laanwj
Copy link
Member

laanwj commented Feb 23, 2022

Currently, the functional test feature_loadblock.py uses it to generate a serialized blockchain (bootstrap.dat), which is then loaded via -loadblock:

Oh that's good to know. I had no idea.

@laanwj
Copy link
Member

laanwj commented Mar 30, 2022

Code review ACK 254a63e

@laanwj laanwj merged commit 171f6f2 into bitcoin:master Mar 30, 2022
@theStack theStack deleted the 202202-contrib-simplify_linearize_script branch March 30, 2022 14:17
sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Apr 3, 2022
@bitcoin bitcoin locked and limited conversation to collaborators Mar 30, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants