-
Notifications
You must be signed in to change notification settings - Fork 38.6k
net: Add Tor v3 hardcoded seeds #21560
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Concept ACK, will review |
74d8869 to
1007a73
Compare
contrib/seeds/nodes_main.txt
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I ran https://gist.github.com/laanwj/b3d7b01ef61ce07c2eff0a72a6b90183 on my node:
- Extra addresses I have that are not on this list:
2g5qfdkn2vvcbqhzcyvyiitg4ceukybxklraxjnu7atlhd22gdwywaid.onion:8333
2jmtxvyup3ijr7u6uvu7ijtnojx4g5wodvaedivbv74w4vzntxbrhvad.onion:8333
37m62wn7dz3uqpathpc4qfmgrbupachj52nt3jbtbjugpbu54kbud7yd.onion:8333
7cgwjuwi5ehvcay4tazy7ya6463bndjk6xzrttw5t3xbpq4p22q6fyid.onion:8333
fjdyxicpm4o42xmedlwl3uvk5gmqdfs5j37wir52327vncjzvtpfv7yd.onion:8333
fzhn4uoxfbfss7h7d6ffbn266ca432ekbbzvqtsdd55ylgxn4jucm5qd.onion:8333
ifdu5qvbofrt4ekui2iyb3kbcyzcsglazhx2hn4wfskkrx2v24qxriid.onion:8333
m7cbpjolo662uel7rpaid46as2otcj44vvwg3gccodnvaeuwbm3anbyd.onion:8333
owjsdxmzla6d7lrwkbmetywqym5cyswpihciesfl5qdv2vrmwsgy4uqd.onion:8333
vi5bnbxkleeqi6hfccjochnn65lcxlfqs4uwgmhudph554zibiusqnad.onion:8333
- Addresses that are on this list, but not on my node:
7pyrpvqdhmayxggpcyqn5l3m5vqkw3qubnmgwlpya2mdo6x7pih7r7id.onion:8333
b64xcbleqmwgq2u46bh4hegnlrzzvxntyzbmucn3zt7cssm7y4ubv3id.onion:8333
fpz6r5ppsakkwypjcglz6gcnwt7ytfhxskkfhzu62tnylcknh3eq6pad.onion:8333
gxo5anvfnffnftfy5frkgvplq3rpga2ie3tcblo2vl754fvnhgorn5yd.onion:8333
itz3oxsihs62muvknc237xabl5f6w6rfznfhbpayrslv2j2ubels47yd.onion:8333
lrjh6fywjqttmlifuemq3puhvmshxzzyhoqx7uoufali57eypuenzzid.onion:8333
opnyfyeiibe5qo5a3wbxzbb4xdiagc32bbce46owmertdknta5mi7uyd.onion:8333
q7kgmd7n7h27ds4fg7wocgniuqb3oe2zxp4nfe4skd5da6wyipibqzqd.onion:8333
sys54sv4xv3hn3sdiv3oadmzqpgyhd4u4xphv4xqk64ckvaxzm57a7yd.onion:8333
tddeij4qigtjr6jfnrmq6btnirmq5msgwcsdpcdjr7atftm7cxlqztid.onion:8333
xqt25cobm5zqucac3634zfght72he6u3eagfyej5ellbhcdgos7t2had.onion:8333
I can connect to all of them except opnyfyeiibe5qo5a3wbxzbb4xdiagc32bbce46owmertdknta5mi7uyd.onion:8333.
- Addresses on both this list and my node:
5g72ppm3krkorsfopcm2bi7wlv4ohhs4u4mlseymasn7g7zhdcyjpfid.onion:8333
ejxefzf5fpst4mg2rib7grksvscl7p6fvjp6agzgfc2yglxnjtxc3aid.onion:8333
rp7k2go3s5lyj3fnj6zn62ktarlrsft2ohlsxkyd7v3e3idqyptvread.onion:8333
It makes things a bit easier if the list is sorted.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you. I pushed a squashme commit to add your nodes and sort the list. I have also adapted the script to print a sorted list.
vasild
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(reviewed up to f3e26a167)
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. ConflictsNo conflicts as of last run. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Concept/approach/code review ACK. I'm ok with squashing.
Right i'll squash the |
vasild
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ede28e76c6ddf5da5518d834c3c3a18ce69cfb6f looks good, modulo the squashing.
I verified that contrib/seeds/generate-seeds.py generates the same src/chainparamsseeds.h as included in this PR.
Also, the added seeds (as reported in the newly added log printout) are the same as the ones in contrib/seeds/nodes_main.txt (serialize from py and deserialize in cpp work as expected).
src/net.cpp
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This printed 1188 lines to the log. I think it is ok.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
AHHH good catch, this is just to debug / test, it's convenient to be able to check if it matches the generated data. I'm not sure there is any point to it for end users 😄 But if you think it's ok we can leave it in.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are also a lot of IP ... mapped to AS0 belongs to new bucket ... messages printed by -debug=net. ~0 on this new printout.
vasild
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
a9ee5683ce4013904f6e53f29e37632969926b0d looks good, modulo the squashing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ACK modulo squash. I'm running the gist (had to update the file perms for it to run) and it's taking a while as I have many (nearly 1835 and increasing) tor v3 addresses; LMK if the output would be useful. A couple nits below to pick/choose/ignore.
The gist script finished. It checked 1836 v3 addresses and returned 1523 of them, sorted. I saved it in a text file if useful. |
|
Update: $ ./src/bitcoin-cli -addressinfo
{
"addresses known": {
"ipv4": 185,
"ipv6": 0,
"torv2": 5597,
"torv3": 2220,
"i2p": 7,
"total": 8009
}
} |
|
Concept ACK. I'll pick up #20648 again after this is merged. |
Maybe! The scope of this PR is just to get things started in that regard, not to add as many nodes as possible. We'd have to decide on how many, and which ones to use; as a guide We could do that kind of process to create a list of I2P and Tor seed nodes for 0.22. The advantage of the dedicated crawler approach to my gist is that it applies some more stringent conditions to the nodes. For example it samples uptime and gets the node's version information. But maybe that can be added, too.
Nice! |
This is necessary now due to parsing change.
Switch from IPv6 slot-based format to more compact and flexible BIP155 format.
a9ee568 to
b2ee8b2
Compare
|
Squashed all the squashme's into the appropriate commits a9ee5683ce4013904f6e53f29e37632969926b0d..b2ee8b207de78f03356905bd60b7b00b6f49c252 The only overall differences are @jonatack's latest comments. |
|
ACK b2ee8b2 Tested hitting |
142e2da net: add I2P seeds to chainparamsseeds (Jon Atack) e01f173 contrib: add a few I2P seed nodes (Jon Atack) ea269c7 contrib: parse I2P addresses in generate-seeds.py (Jon Atack) Pull request description: Follow-up to #21560 that updated the fixed seeds infra for BIP155 addresses and then added Tor v3 ones: - Update contrib/generate-seeds.py to parse I2P addresses - Add a few I2P nodes to contrib/seeds/nodes_main.txt - Run generate-seeds.py and add the I2P seeds to chainparamsseeds.h Reviewers, see contrib/seeds/README.md for more info and feel free to use the following CLI one-liner to check for and propose additional seeds for contrib/seeds/nodes_main.txt. You can also see how many I2P peers your node knows with cli -addrinfo. ```rake bitcoin-cli getnodeaddresses 0 | jq '.[] | (select(.address | contains(".b32.i2p"))) | .address' | sort ``` I verified the I2P addresses are correctly BIP155-serialized/deserialized by building with all seeds removed from chainparamsseeds.h except those added here, restarting with `-datadir=newdir -dnsseed=0` and running rpc ` getnodeaddresses 0` that initially returns only the new I2P addresses. ACKs for top commit: laanwj: ACK 142e2da vasild: ACK 142e2da Tree-SHA512: 040576012d5f1f034e2bd566ad654a6fdfd8ff7f6b12fa40c9fda1e948ebf8417fcea64cfc14938a41439370aa4669bab3e97274f9d4f9a6906fa9520afa9cf8
6fe0516 contrib: add torv3 seed nodes for testnet, drop v2 ones (Jon Atack) Pull request description: Replace the ancient (2015) Tor V2 hardcoded seeds with new Tor V3 ones. This needs to be done before 0.22 to make sure onion-only testnet nodes can still connect to the network. Continues #21560. Ways to test: - Re-generate ` src/chainparamsseeds.h` with `cd contrib/seeds && python3 generate-seeds.py . > ../../src/chainparamsseeds.h`, check if git tree stays the same. - Create a new testnet node with `bitcoind -testnet -onlynet=onion -proxy=127.0.0.1:9050` (or delete `~/.bitcoin/testnet3/peers.dat`), check if it is able to connect to the network and get blocks. - Check if the addresses are connectable for ex.: ```python3 #!/usr/bin/env python3 import subprocess with open('contrib/seeds/nodes_test.txt') as f: for line in (line for line in (line.rstrip().split('#', 1)[0] for line in f) if line): subprocess.call(["nc", "-v", "-x", "127.0.0.1:9050", "-z"] + line.split(':')) ``` Thanks to jonatack for providing the list. ACKs for top commit: jonatack: ACK 6fe0516 Tree-SHA512: 61bfdb44dfab9d02b75e5cb06c089a3b1a1fe7134875e1d09166c4116e961d809aa25422fe03f068876e9423b571ecc4a0c7a7eeacba4aac3b2768717f3ee6d6
4aec935 net: Deserialize hardcoded seeds from BIP155 blob (Fuzzbawls) a245b2f contrib: generate-seeds.py generates output in BIP155 format (Fuzzbawls) Pull request description: Adapted from bitcoin#21560, this adds support for Tor v3 hardcoded seed nodes. Upstream PR description: > * Add a few hardcoded seeds for TorV3 > * As the bitcoin-seeder doesn't collect TorV3 addresses yet, I have extracted these from my own node using a [script](https://gist.github.com/laanwj/b3d7b01ef61ce07c2eff0a72a6b90183) and added them manually. This is intended to be a temporary stop gap until 22.0's seeds update. > * Change hardcoded seeds to variable length BIP155 binary format. > * It is stored as a single serialized blob in a byte array, instead of pseudo-IPv6 address slots. This is more flexible and, assuming most of the list is IPv4, more compact. > * Only the (networkID, addr, port) subset (CService). Services and time are construed on the fly as before. > * Change input format for nodes_*.txt. > * Drop legacy 0xAABBCCDD format for IPv4. It is never generated by makeseeds.py. > * Stop interpreting lack of port as default port, interpret it as 'no port', to accomodate I2P and other port-less protocols (not handled in this PR). An explicit port is always generated by makeseeds.py so in practice this makes no difference right now. Marked as WIP for now because we need more Tor v3 nodes on mainnet for this to really be of use to end users, but this is currently testable on testnet as we already have a number of Tor v3 capable nodes there (which have been added to the hardcoded seeds here) ACKs for top commit: random-zebra: ACK 4aec935 furszy: ACK 4aec935 Tree-SHA512: 5abf2faa49ca511001589d950fa95a6f4c346a391ab0b31b3137b962765dc5a5ba343aa4c7b7fa0d7be931d4ec4802d9c09ec9b5dd2ef90143d4623162da14af
Closes #20239 and mitigates my node's problem in #21351.
Add a few hardcoded seeds for TorV3
Change hardcoded seeds to variable length BIP155 binary format.
Change input format for
nodes_*.txt.0xAABBCCDDformat for IPv4. It is never generated bymakeseeds.py.makeseeds.pyso in practice this makes no difference right now.A follow-up to this PR could do the same for I2P.