Skip to content

Conversation

@jonatack
Copy link
Member

@jonatack jonatack commented Feb 27, 2025

In makeseeds.py:

  • fix the user agent regex (by laanwj)
  • fix the I2P column header spacing
  • update the regex (it was also not updated for the previous release)
  • update MIN_BLOCKS (4320 blocks/month * ~6.5 months) and add README documentation to remember to update it
  • further robustness/standardness/consistency improvements to the regexes (by l0rinc)

Add signet and testnet4 seeds to the README and to generate-seeds.py

Update the fixed seeds in src/chainparamsseeds.h

In kernel/chainparams.cpp:

  • add signet fixed seeds if default network
  • remove hardcoded signet seeds

Update contrib/seeds/.gitignore with signet and testnet4

The previous 2 seeds updates were #30008 and #30695.

mainnet:

$ contrib/seeds$ python3 makeseeds.py -a asmap-filled.dat -s seeds_main.txt > nodes_main.txt

Loading asmap database "asmap-filled.dat"…Done.
Loading and parsing DNS seeds…Done.
  IPv4   IPv6  Onion    I2P  CJDNS Pass                                               
 17252   3630  21079   3095     12 Initial
 17252   3630  21079   3095     12 Skip entries with invalid address
  8444   1742  14607   2330     10 After removing duplicates
  8194   1691  14321   2102     10 Enforce minimal number of blocks
  7838   1578  14321   2102     10 Require service bit 1
  6802   1326  14321   2102     10 Require minimum uptime
  6762   1321  14320   2102     10 Require a known and recent user agent
  6762   1321  14320   2102     10 Filter out hosts with multiple bitcoin ports
   512    485    512    512     10 Look up ASNs and limit results per ASN and per net

signet:

$ contrib/seeds$ python3 makeseeds.py -a asmap-filled.dat -s seeds_signet.txt -m 237800 > nodes_signet.txt


Loading asmap database "asmap-filled.dat"…Done.
Loading and parsing DNS seeds…Done.
  IPv4   IPv6  Onion    I2P  CJDNS Pass                                               
   110     47     63      9      4 Initial
   110     47     63      9      4 Skip entries with invalid address
   110     47     63      9      4 After removing duplicates
    83     31     58      9      4 Enforce minimal number of blocks
    83     31     58      9      4 Require service bit 1
    83     31     57      9      4 Require minimum uptime
    83     31     57      9      4 Require a known and recent user agent
    83     31     57      7      4 Filter out hosts with multiple bitcoin ports
    42     30     57      7      4 Look up ASNs and limit results per ASN and per net

testnet:

$ contrib/seeds$ python3 makeseeds.py -a asmap-filled.dat -s seeds_test.txt > nodes_test.txt

Loading asmap database "asmap-filled.dat"…Done.
Loading and parsing DNS seeds…Done.
  IPv4   IPv6  Onion    I2P  CJDNS Pass                                               
   204     73     96     11      5 Initial
   204     73     96     11      5 Skip entries with invalid address
   204     73     96     11      5 After removing duplicates
   204     73     96     11      5 Enforce minimal number of blocks
   204     73     96     11      5 Require service bit 1
   195     69     87      9      5 Require minimum uptime
   193     69     87      9      5 Require a known and recent user agent
   193     69     87      9      5 Filter out hosts with multiple bitcoin ports
    79     39     87      9      5 Look up ASNs and limit results per ASN and per net

testnet4

$ contrib/seeds$ python3 makeseeds.py -a asmap-filled.dat -s seeds_testnet4.txt -m 72600 > nodes_testnet4.txt

Loading asmap database "asmap-filled.dat"…Done.
Loading and parsing DNS seeds…Done.
  IPv4   IPv6  Onion    I2P  CJDNS Pass                                               
   149    115     69     11      4 Initial
   149    115     69     11      4 Skip entries with invalid address
   149    115     69     11      4 After removing duplicates
   104     75     52      7      4 Enforce minimal number of blocks
   104     75     52      7      4 Require service bit 1
   100     73     51      7      4 Require minimum uptime
   100     73     51      7      4 Require a known and recent user agent
   100     73     51      7      4 Filter out hosts with multiple bitcoin ports
    43     46     51      7      4 Look up ASNs and limit results per ASN and per net

@DrahtBot
Copy link
Contributor

DrahtBot commented Feb 27, 2025

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/31960.

Reviews

See the guideline for information on the review process.

Type Reviewers
ACK l0rinc, achow101, laanwj
Concept ACK 1440000bytes

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #31991 (RFC: Add operator""_uint256 compile-time user-defined literal by l0rinc)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@glozow glozow added this to the 29.0 milestone Mar 3, 2025
@1440000bytes

This comment was marked as abuse.

@jonatack jonatack force-pushed the 2025-02-update-dns-seeds branch 2 times, most recently from 8c054bd to b2dc445 Compare March 3, 2025 18:19
@jonatack
Copy link
Member Author

jonatack commented Mar 3, 2025

Updated to take @laanwj's review feedback and re-ran the seeds scripts.

@laanwj
Copy link
Member

laanwj commented Mar 3, 2025

Code review ACK b2dc4458ac86c0a633ca3eb87827ac2731993f97 on the script changes

mainnet

Loading asmap database "asmap-filled.dat"…Done.
Loading and parsing DNS seeds…Done.
  IPv4   IPv6  Onion    I2P  CJDNS Pass
 17173   3638  21303   3116     12 Initial
 17173   3638  21303   3116     12 Skip entries with invalid address
  8378   1755  14661   2343     10 After removing duplicates
  8337   1736  14656   2343     10 Enforce minimal number of blocks
  7986   1612  14654   2343     10 Require service bit 1
  6963   1365  14644   2343     10 Require minimum uptime
  6917   1359  14643   2343     10 Require a known and recent user agent
  6917   1359  14643   2343     10 Filter out hosts with multiple bitcoin ports
   512    502    512    512     10 Look up ASNs and limit results per ASN and per net

testnet

Loading asmap database "asmap-filled.dat"…Done.
Loading and parsing DNS seeds…Done.
  IPv4   IPv6  Onion    I2P  CJDNS Pass
   204     74     95     11      5 Initial
   204     74     95     11      5 Skip entries with invalid address
   204     74     95     11      5 After removing duplicates
   204     74     95     11      5 Enforce minimal number of blocks
   204     74     95     11      5 Require service bit 1
   197     68     87      9      5 Require minimum uptime
   195     68     87      9      5 Require a known and recent user agent
   195     68     87      9      5 Filter out hosts with multiple bitcoin ports
    78     38     87      9      5 Look up ASNs and limit results per ASN and per net

i have re-run the steps in an attempt to check the output, but the lack of determinism makes it hard to say anything. All that could be concluded is that the diff is smaller from the previous state to the current one, to the one between the current one and my re-run:

$ git show HEAD --shortstat nodes_main.txt
commit b2dc4458ac86c0a633ca3eb87827ac2731993f97 (HEAD)
 1 file changed, 1618 insertions(+), 1627 deletions(-)
$ git diff --shortstat nodes_main.txt
 1 file changed, 419 insertions(+), 418 deletions(-)

$ git show HEAD --shortstat nodes_test.txt
commit b2dc4458ac86c0a633ca3eb87827ac2731993f97 (HEAD)
 1 file changed, 128 insertions(+), 222 deletions(-)
$ git diff --shortstat nodes_test.txt
(no output)

The testnet output does match 100%.

@jonatack
Copy link
Member Author

jonatack commented Mar 3, 2025

Yes, for the avoidance of doubt, I ran the script with no manual curation (as done previously from v21 to v27).

@DrahtBot DrahtBot mentioned this pull request Mar 3, 2025
@achow101
Copy link
Member

achow101 commented Mar 3, 2025

Could also do testnet4, my seeder publishes a seeds.txt.gz at https://testnet4.achownodes.xyz/seeds.txt.gz. And also signet: https://signet.achownodes.xyz/seeds.txt.gz

@jonatack jonatack force-pushed the 2025-02-update-dns-seeds branch from b2dc445 to 959d662 Compare March 4, 2025 21:50
@jonatack
Copy link
Member Author

jonatack commented Mar 4, 2025

Could also do testnet4, my seeder publishes a seeds.txt.gz at testnet4.achownodes.xyz/seeds.txt.gz. And also signet: signet.achownodes.xyz/seeds.txt.gz

Done. Also updated MIN_BLOCKS and added a reminder to the README.

@jonatack jonatack changed the title seeds: update makeseeds regex and DNS fixed seeds seeds: add signet/testnet4, update makeseeds regex, minblocks, fixed seeds Mar 4, 2025
Copy link
Contributor

@l0rinc l0rinc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have only reviewed the regexes, please see my suggestions.

Regexes
PATTERN_IPV4 = re.compile(r"^(([0-2]?\d{1,2})\.([0-2]?\d{1,2})\.([0-2]?\d{1,2})\.([0-2]?\d{1,2})):(\d{1,5})$")
PATTERN_IPV6 = re.compile(r"^\[([\da-f:]+)]:(\d{1,5})$", re.IGNORECASE)
PATTERN_ONION = re.compile(r"^([a-z2-7]{56}\.onion):(\d{1,5})$")
PATTERN_I2P = re.compile(r"^([a-z2-7]{52}\.b32\.i2p):(\d{1,5})$")
PATTERN_AGENT = re.compile(
    r"^/Satoshi:("
    r"0\.14\.([0-3]|99)"
    r"|0\.15\.([0-2]|99)"
    r"|0\.16\.([0-3]|99)"
    r"|0\.17\.(0\.1|[0-2]|99)"
    r"|0\.18\.([01]|99)"
    r"|0\.19\.([0-2]|99)"
    r"|0\.20\.([0-2]|99)"
    r"|0\.21\.([0-2]|99)"
    r"|22\.([01]|99)\.0"
    r"|23\.([01]|99)\.0"
    r"|24\.([0-2]|99)\.[01]"
    r"|25\.([0-2]|99)\.0"
    r"|26\.([01]|99)\.0"
    r"|27\.([01]|99)\.0"
    r"|28\.(0|99)\.0"
    r")")

@achow101
Copy link
Member

achow101 commented Mar 5, 2025

While src/chainparamsseeds.h now has chainparams_seed_signet, it is not used anywhere. SigNetParams does not set vFixedSeeds as the other ones do. It probably should, but only when using the default testnet, as we do for the DNS seeds.

@jonatack jonatack force-pushed the 2025-02-update-dns-seeds branch from e482f33 to ff2a5ae Compare March 6, 2025 19:58
@jonatack
Copy link
Member Author

jonatack commented Mar 6, 2025

Thanks for the reviews @l0rinc and @achow101. Updated to take @achow101's suggestion, added a commit by @l0rinc with his review feedback (makeseeds output appears unchanged by the robustness/consistency changes to the regexes), updated .gitignore in /contrib/seeds, re-ran the scripts, repushed and updated the PR description.

@jonatack jonatack force-pushed the 2025-02-update-dns-seeds branch from ff2a5ae to 9ef218c Compare March 6, 2025 20:14
@jonatack
Copy link
Member Author

jonatack commented Mar 6, 2025

Re-pushed to add a missing escape to the regexes commit that I had overlooked.

laanwj and others added 7 commits March 6, 2025 14:50
We shouldn't have | at the end of the last clause, as this will make it match
the empty string too (so effectively everything starting with Satoshi: matches).

While doing this, put the | at the beginning of every line of regex rather than
the end, to make it easier to update in the future without accidentally running
into this problem again.
@achow101
Copy link
Member

achow101 commented Mar 6, 2025

Needs rebase

@jonatack jonatack force-pushed the 2025-02-update-dns-seeds branch from 9ef218c to 89fd9a4 Compare March 6, 2025 20:52
@jonatack
Copy link
Member Author

jonatack commented Mar 6, 2025

Rebased

@achow101
Copy link
Member

achow101 commented Mar 6, 2025

ACK 89fd9a4a87d772798671da18538e45f0272365e4

@DrahtBot DrahtBot requested a review from laanwj March 6, 2025 21:12
@jonatack jonatack force-pushed the 2025-02-update-dns-seeds branch from 89fd9a4 to f0b6597 Compare March 6, 2025 21:40