Skip to content

Conversation

@marcoagner
Copy link
Contributor

Fixes #13579.
I think one possible solution is to check for _transport.is_closing() in the lambda before sending a message (compatible with Python 3.4 too). Let me know if I missed any side effects this introduces.

…nsport

- checks if  _transport.is_closing() (added in python3.4.4/python3.5.1)
before attempting to send messages on P2PConnection's send_message
method.
@maflcko
Copy link
Member

maflcko commented Jul 19, 2018

utACK ea5340c. Indeed, we shouldn't write to a transport that is closing. I don't see any side effects other than preventing the raised exceptions.

@maflcko maflcko added the Tests label Jul 19, 2018
@maflcko maflcko merged commit ea5340c into bitcoin:master Jul 19, 2018
maflcko pushed a commit that referenced this pull request Jul 19, 2018
…on closing transport

ea5340c tests: fixes mininode's P2PConnection sending messages on closing transport (marcoagner)

Pull request description:

  Fixes #13579.
  I think one possible solution is to check for [`_transport.is_closing()`](https://docs.python.org/3.4/library/asyncio-protocol.html#asyncio.BaseTransport.is_closing) in the lambda before sending a message (compatible with Python 3.4 too). Let me know if I missed any side effects this introduces.

Tree-SHA512: cab46f81dccfec7b4460fda478a617845564520694449a9e85bf8a5f1e75f35f52cafd7c64966712c3d6c29956344d5a9dbad8851424f061eb3748bc621b900b
@maflcko maflcko added this to the 0.17.0 milestone Jul 19, 2018
@marcoagner marcoagner deleted the fix_mininode_p2pconnection_send_message branch July 19, 2018 15:30
@domob1812
Copy link
Contributor

domob1812 commented Jul 23, 2018

Note that this seems to break with Debian Jessie (oldstable), which has only Python 3.4.2. I don't know if compatibility with Debian oldstable is a goal or not.

@fanquake
Copy link
Member

@domob1812 If this has broken the tests on a system using Python 3.4+ then that's a problem, as 3.4 is our minimum supported version. Please open an issue/PR with any more details.

@domob1812
Copy link
Contributor

Thanks for clarifying that this is actually an issue we care about! Created #13745 for it.

domob1812 added a commit to domob1812/namecoin-core that referenced this pull request Jul 23, 2018
bitcoin/bitcoin#13715 broke compatibility
with Debian Jessie.  For now, this includes a workaround for this,
although we might remove that later and drop compatibility ourselves.
See also bitcoin/bitcoin#13745.

Note that the new rpc_psbt.py test is broken upstream, see
bitcoin/bitcoin#13738.

Conflicts:
	src/rpc/rawtransaction.h
	src/wallet/rpcwallet.cpp
domob1812 added a commit to domob1812/bitcoin that referenced this pull request Jul 23, 2018
bitcoin#13715 introduced a new check
for _transport.is_closing() in mininode's P2PConnection's.  This function
is only available from Python 3.4.4, though, while Bitcoin is supposed
to support all Python 3.4 versions.

In this change, we make the check conditional on is_closing() being
available.  If it is not, then we revert to the behaviour before the
check was introduced; this means that
bitcoin#13579 is not fixed for old
systems, but at least the tests work as they used to do before.

This includes a small refactoring from a one-line lambda to an
inline function, because this makes the code easier to read with more
and more conditions being added.

Fixes bitcoin#13745.
maflcko pushed a commit that referenced this pull request Jul 23, 2018
… available

64b9f27 Skip is_closing() check when not available. (Daniel Kraft)

Pull request description:

  #13715 introduced a new check for `_transport.is_closing()` in mininode's `P2PConnection`'s.  This function is [only available from Python 3.4.4](https://docs.python.org/3.4/library/asyncio-protocol.html#asyncio.BaseTransport.is_closing), though, while Bitcoin Core is supposed to support all Python 3.4 versions.

  In this change, we make the check conditional on `is_closing` being available.  If it is not, then we revert to the behaviour before the check was introduced; this means that #13579 is not fixed for old systems, but at least the tests work as they used to do before.

  This includes a small refactoring from a one-line lambda to an inline function, because this makes the code easier to read with more and more conditions being added.

  Fixes #13745.

Tree-SHA512: 15be03b8b49c40a946c5b354c5974858d14dc46283ad48ee25d9e269377077ce741c6b379b3f6581ab981cb65be799809afbb99da278caaa2d8d870fa4fb748f
Bushstar pushed a commit to Bushstar/Feathercoin that referenced this pull request Jul 24, 2018
bitcoin/bitcoin#13715 introduced a new check
for _transport.is_closing() in mininode's P2PConnection's.  This function
is only available from Python 3.4.4, though, while Bitcoin is supposed
to support all Python 3.4 versions.

In this change, we make the check conditional on is_closing() being
available.  If it is not, then we revert to the behaviour before the
check was introduced; this means that
bitcoin/bitcoin#13579 is not fixed for old
systems, but at least the tests work as they used to do before.

This includes a small refactoring from a one-line lambda to an
inline function, because this makes the code easier to read with more
and more conditions being added.

Fixes bitcoin/bitcoin#13745.
HashUnlimited pushed a commit to HashUnlimited/chaincoin that referenced this pull request Sep 12, 2018
bitcoin#13715 introduced a new check
for _transport.is_closing() in mininode's P2PConnection's.  This function
is only available from Python 3.4.4, though, while Bitcoin is supposed
to support all Python 3.4 versions.

In this change, we make the check conditional on is_closing() being
available.  If it is not, then we revert to the behaviour before the
check was introduced; this means that
bitcoin#13579 is not fixed for old
systems, but at least the tests work as they used to do before.

This includes a small refactoring from a one-line lambda to an
inline function, because this makes the code easier to read with more
and more conditions being added.

Fixes bitcoin#13745.
jfhk pushed a commit to jfhk/bitcoin that referenced this pull request Nov 14, 2018
bitcoin#13715 introduced a new check
for _transport.is_closing() in mininode's P2PConnection's.  This function
is only available from Python 3.4.4, though, while Bitcoin is supposed
to support all Python 3.4 versions.

In this change, we make the check conditional on is_closing() being
available.  If it is not, then we revert to the behaviour before the
check was introduced; this means that
bitcoin#13579 is not fixed for old
systems, but at least the tests work as they used to do before.

This includes a small refactoring from a one-line lambda to an
inline function, because this makes the code easier to read with more
and more conditions being added.

Fixes bitcoin#13745.
furszy pushed a commit to furszy/bitcoin-core that referenced this pull request Jun 28, 2021
bitcoin#13715 introduced a new check
for _transport.is_closing() in mininode's P2PConnection's.  This function
is only available from Python 3.4.4, though, while Bitcoin is supposed
to support all Python 3.4 versions.

In this change, we make the check conditional on is_closing() being
available.  If it is not, then we revert to the behaviour before the
check was introduced; this means that
bitcoin#13579 is not fixed for old
systems, but at least the tests work as they used to do before.

This includes a small refactoring from a one-line lambda to an
inline function, because this makes the code easier to read with more
and more conditions being added.

Fixes bitcoin#13745.
random-zebra added a commit to PIVX-Project/PIVX that referenced this pull request Jun 28, 2021
…t framework update)

7b04c0a [Test] Clean duplicate connections creation in mining_pos_coldStaking.py (furszy)
15a799e [Test] MAX_PROTOCOL_MESSAGE_LENGTH PIVXified. (furszy)
0aedf35 [tests] Don't import asyncio to test magic bytes (John Newbery)
9bb5309 Refactor resource exhaustion test (Troy Giorshev)
589a780 Fix "invalid message size" test (Troy Giorshev)
8a0c12b Move size limits to module-global (Troy Giorshev)
b3391c5 Remove two unneeded tests (Troy Giorshev)
1404e68 test: Add various low-level p2p tests (furszy)
8aaf7e1 test: Remove fragile assert_memory_usage_stable (MarcoFalke)
f68e22c [Test] p2p_invalid_messages.py do not change msg.command in test_command and raise sync_with_ping timeout (furszy)
c851c0b Test framework: Wait for verack by default on every new connection (furszy)
c02e9a0 [Test] move framework subversion to string (furszy)
3472a39 Replace coroutine with async def in p2p_invalid_messages.py (Elichai Turkel)
33c7b19 net_processing: align msg checksum check properly. (furszy)
7f71b1b Hash P2P messages as they are received instead of at process-time (furszy)
215a638 test: Skip flaky p2p_invalid_messages test on macOS (Fabian Jahr)
c11f565 qa: Make swap_magic_bytes in p2p_invalid_messages atomic (MarcoFalke)
be979ad test: Fix race in p2p_invalid_messages (MarcoFalke)
6a72f0c qa: Add tests for invalid message headers (MarcoFalke)
51ddd3d Introduce p2p_invalid_messages.py test (furszy)
55a37b5 net: fix missing jump line in "Oversized message from peer.." error. (furszy)
0edfce5 test_node: get_mem_rss fixups (MarcoFalke)
6f21213 tests: add P2PConnection.send_raw_message (James O'Beirne)
ae68c6e tests: add utility to assert node memory usage hasn't increased (James O'Beirne)
8469afe test: forward timeouts properly in send_blocks_and_test (James O'Beirne)
db28a53 Skip is_closing() check when not available. (Daniel Kraft)
be9dacb tests: fixes mininode's P2PConnection sending messages on closing transport (marcoagner)
53599c8 qa: Avoid start/stop of the network thread mid-test (furszy)
688190c [qa] mininode: Expose connection state through is_connected (MarcoFalke)

Pull request description:

  Part of the deep and long net and ser work that I'm doing (and Tor v3 onion addresses support). Friend of #2359.

  Focused on the end goal of implementing the `p2p_invalid_messages` functional test which validates that invalid msg headers, msg types, oversized payloads and inventory msgs aren't accepted nor can cause a resource exhaustion. And an extra covered scenario, in `p2p_invalid_tx.py`, for the orphan pool overflow.

  Plus, to get up to the goal, had to work over the functional test framework as well.

  So.. adapted list:

  * bitcoin#9045.
  * bitcoin#13512.
  * bitcoin#13517.
  * bitcoin#13715.
  * bitcoin#13747.
  * bitcoin#14456.
  * bitcoin#14522.
  * bitcoin#14672.
  * bitcoin#14693.
  * bitcoin#14812.
  * bitcoin#15246.
  * bitcoin#15330.
  * bitcoin#15697.
  * bitcoin#16445.
  * bitcoin#17931.
  * bitcoin#17469.
  * bitcoin#18628 (only `p2p_invalid_tx.py` and `p2p_invalid_messages.py`. We don't support the other tests yet).
  * bitcoin#19177.
  * bitcoin#19264.

ACKs for top commit:
  random-zebra:
    utACK 7b04c0a and merging...

Tree-SHA512: 48d1f1a6acc24a9abce033fbf1f281ba4392147da39d118a694a09d63c3e0610cc1a29d711b434b16cc0d0e030dd9f1a89843870091b6999b862b9ab18a20679
UdjinM6 pushed a commit to UdjinM6/dash that referenced this pull request Jul 1, 2021
…ssages on closing transport

ea5340c tests: fixes mininode's P2PConnection sending messages on closing transport (marcoagner)

Pull request description:

  Fixes bitcoin#13579.
  I think one possible solution is to check for [`_transport.is_closing()`](https://docs.python.org/3.4/library/asyncio-protocol.html#asyncio.BaseTransport.is_closing) in the lambda before sending a message (compatible with Python 3.4 too). Let me know if I missed any side effects this introduces.

Tree-SHA512: cab46f81dccfec7b4460fda478a617845564520694449a9e85bf8a5f1e75f35f52cafd7c64966712c3d6c29956344d5a9dbad8851424f061eb3748bc621b900b
UdjinM6 pushed a commit to UdjinM6/dash that referenced this pull request Jul 1, 2021
…hen not available

64b9f27 Skip is_closing() check when not available. (Daniel Kraft)

Pull request description:

  bitcoin#13715 introduced a new check for `_transport.is_closing()` in mininode's `P2PConnection`'s.  This function is [only available from Python 3.4.4](https://docs.python.org/3.4/library/asyncio-protocol.html#asyncio.BaseTransport.is_closing), though, while Bitcoin Core is supposed to support all Python 3.4 versions.

  In this change, we make the check conditional on `is_closing` being available.  If it is not, then we revert to the behaviour before the check was introduced; this means that bitcoin#13579 is not fixed for old systems, but at least the tests work as they used to do before.

  This includes a small refactoring from a one-line lambda to an inline function, because this makes the code easier to read with more and more conditions being added.

  Fixes bitcoin#13745.

Tree-SHA512: 15be03b8b49c40a946c5b354c5974858d14dc46283ad48ee25d9e269377077ce741c6b379b3f6581ab981cb65be799809afbb99da278caaa2d8d870fa4fb748f
UdjinM6 pushed a commit to UdjinM6/dash that referenced this pull request Jul 2, 2021
…ssages on closing transport

ea5340c tests: fixes mininode's P2PConnection sending messages on closing transport (marcoagner)

Pull request description:

  Fixes bitcoin#13579.
  I think one possible solution is to check for [`_transport.is_closing()`](https://docs.python.org/3.4/library/asyncio-protocol.html#asyncio.BaseTransport.is_closing) in the lambda before sending a message (compatible with Python 3.4 too). Let me know if I missed any side effects this introduces.

Tree-SHA512: cab46f81dccfec7b4460fda478a617845564520694449a9e85bf8a5f1e75f35f52cafd7c64966712c3d6c29956344d5a9dbad8851424f061eb3748bc621b900b
UdjinM6 pushed a commit to UdjinM6/dash that referenced this pull request Jul 2, 2021
…hen not available

64b9f27 Skip is_closing() check when not available. (Daniel Kraft)

Pull request description:

  bitcoin#13715 introduced a new check for `_transport.is_closing()` in mininode's `P2PConnection`'s.  This function is [only available from Python 3.4.4](https://docs.python.org/3.4/library/asyncio-protocol.html#asyncio.BaseTransport.is_closing), though, while Bitcoin Core is supposed to support all Python 3.4 versions.

  In this change, we make the check conditional on `is_closing` being available.  If it is not, then we revert to the behaviour before the check was introduced; this means that bitcoin#13579 is not fixed for old systems, but at least the tests work as they used to do before.

  This includes a small refactoring from a one-line lambda to an inline function, because this makes the code easier to read with more and more conditions being added.

  Fixes bitcoin#13745.

Tree-SHA512: 15be03b8b49c40a946c5b354c5974858d14dc46283ad48ee25d9e269377077ce741c6b379b3f6581ab981cb65be799809afbb99da278caaa2d8d870fa4fb748f
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test_runner sometimes fail due to "socket.send() raised exception" in stderr

5 participants