Skip to content

Conversation

@vincenzopalazzo
Copy link

This is one of my attempts to make less painful the #23127 and try to bring this simple refactoring in.

Issue related #23119

PR 1/N I think there are a couple of equal size later

@DrahtBot DrahtBot added the Tests label Jul 28, 2022
@DrahtBot
Copy link
Contributor

DrahtBot commented Jul 29, 2022

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #23502 (wallet: Avoid underpaying transaction fees when signing taproot spends by achow101)
  • #23127 (tests: Use test framework utils where possible by vincenzopalazzo)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@fanquake
Copy link
Member

test/functional/interface_zmq.py:367: error: unexpected indent  [syntax]
Found 1 error in 1 file (errors prevented further checking)
/tmp/cirrus-ci-build/test/functional/interface_zmq.py:367: unexpected indent at "assert_raises_rpc_error(-8, "Verbose results cannot contain mempool sequence values.", self.nodes[0].getrawmempool, True, True)"
Python dead code detection found some issues

@vincenzopalazzo vincenzopalazzo force-pushed the macros/assert_promotion branch from 78ef523 to 97ff626 Compare July 29, 2022 09:26
@vincenzopalazzo
Copy link
Author

vincenzopalazzo commented Jul 29, 2022

Thanks! Cherry-pick problem with an older version of the test/functional/interface_zmq.py file, lets see if the CI catch other stuff

Copy link
Contributor

@stickies-v stickies-v left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approach ACK ad9dcabd5d9cef3f16accbc9dc1572cd82a99081

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

== and is are not entirely equivalent in Python, with both differences in evaluation speed and outcome. Even though I think it's fine to use the equality operator here, I think the proper way to do it would be to add assert_is and/or assert_is_none functions (e.g. unittest also has these: https://docs.python.org/3/library/unittest.html#assert-methods)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'm getting confused here :)

We need to converge on some code style here otherwise we cause only confusion #23127 (comment)

cc @MarcoFalke

* feature_taproot: Introduce another assert called assert_true. 

* interface_zmq.py: Remove assert in favor of test utils functions

* feature_taproot: Fixed imports

* mining_getblocktemplate_longpoll.py: Replace assert with assert_equal

* p2p_addrfetch.py: Replace assert with assert_equal

* p2p_blockfilters.py: Replace assert with test utils function

* test framework: Adding possibility to specify messages in the assert functions

* feature_taproot.py: Use the err_msg introduced in the assert framework.

* feature_taproot.py: Use the err_msg introduced in the assert framework.

* feature_taproot.py: Use the err_msg introduced in the assert framework.
@vincenzopalazzo
Copy link
Author

Thanks @stickies-v, you remember me that we have a pending decision on the API that we are using right now #23127 (comment) maybe we can discuss it here!

@vincenzopalazzo vincenzopalazzo force-pushed the macros/assert_promotion branch from ad9dcab to 640edd9 Compare August 1, 2022 21:19
Copy link
Contributor

@stickies-v stickies-v left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't want to bikeshed over this too much, it's not a big deal. But my view (and it has changed a bit after reading the previous issues/PRs) is that:

  • we check values for being (not) None quite often. Even though == comparison would work fine in probably all cases, it's not the Python way of doing things and given how often this is used I feel adding a assert_is_none() and assert_is_not_none() function is warranted. Since generally nothing except for None should be compared through is, I'd say we can leave out assert_is() and assert_is_none() (which I suggested earlier) until we actually need it (probably never).
  • even though boolean assert_true() and assert_false() functions can have their merit, upon re-reading I don't see any in the current PR because we currently only ever pass it booleans. In all the cases where assert_true() is used, a simple assert <expression>, <err_msg> is functionally equivalent, shorter, and without the need to define assert_true(). In all places where assert_true() is used, the logs would just show False is not True, which does not help. I believe this is what @MarcoFalke was referring to, and I agree with him for this scenario (but not for the assert_equal() on which the comment thread was based). Having a boolean assertion function is useful as soon as you need to pass non-boolean types to it. If we do decide to implement an assert_true(expr, err_msg) function, it should be implemented to evaluate bool(expr) is not True instead of just expr is not True (as commented), in which case it offers benefit over assert_equal(bool(expr), True) because we can now log the value of expr instead of bool(expr). But again, it looks like we currently don't have a need for this.
  • From my reading of previous issues, it seems like there is no reason to necessarily phase out all usage of plain assert statements here - the only reason to add bespoke assert_...() functions was to make it easier to include error messages. As such, I think it's fine to keep plain assert statements where it's easier until we have a clear reason (please tell me if I've missed that) to do so.



def assert_true(thing, err_msg=None):
if thing is not True:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These functions are generally (e.g. https://docs.python.org/3/library/unittest.html#unittest.TestCase.assertTrue) used to do a boolean comparison, which makes it functionally different than using assert_equal(expr, True)

It wouldn't make a functional difference in the places where it's currently used (where a simple assert would suffice), but otherwise, this is how I think it should be done:

Suggested change
if thing is not True:
if bool(thing) is not True:

Copy link
Author

@vincenzopalazzo vincenzopalazzo Aug 2, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

with this suggestion, you will create an error a runtime and no static checker can find it. We can use the only good thing of python language to write something like that

def assert_true(thing: bool, err_msg=None):
    if thing is not True:

In this way, the editor is able to catch the type mismatch! the cast of a type at runtime should be an antipattern in all modern language

Copy link
Contributor

@stickies-v stickies-v Aug 2, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only way bool(thing) throws an error is if thing is of a custom type that explicitly overrides __bool__() to return something other than True or False, which is... really bad code. I don't think we need to handle that, and we can easily observe if it happens from the stacktrace.

Even though I'm a fan of type hints, I don't think they change anything here - afaik we don't use tooling like mypy that would enforce them? It also wouldn't do the boolean comparison which was kinda my point: you don't want assert_true() to just verify if something is equal to True/False (assert_equal can do that), it's helpful if you need to check if the boolean of an object would evaluate to True/False. For example:

result = some_func()  # some_func returns 0 if it failed, an int between 1-5 if successful
assert_equal(result, True) # this would not work, because e.g. `3 == True` would fail
assert_equal(bool(result), True)  # this would work, but you can't inspect `result` when the assertion fails
assert_true(result)  # this would work if `assert_true` evaluates bool(result), because `bool(3) is True` would pass

if(result):
    do_something_else()

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But to reiterate, I'm advocating against implementing assert_true() at this moment, so I'll leave it here

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right! The thing is worst than the exception because happens something like that

➜  ~ python3
Python 3.8.10 (default, Jun 22 2022, 20:18:18) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> bool(12)
True
>>> bool("Alibaba")
True
>>> 

This can not hide bugs?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think to make all happy here is to use the assert_equal everywhere it is possible and move on.

This is code that I wrote one year ago, so I do not remember why I insert assert_true, but if I was not drunk I think because somewhere there is some comparison with custom error messages that check a boolean condition.

Anyway, I think if you agree we can move all on assert_equal and move on.

you don't want assert_true() to just verify if something is equal to True/False

To sustain your thesis, I'm not able to see any real use case other than toy example of how this can be used

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is that worse? You just need to use the right tool for the job. If you need to compare equality, you use assert_equal. If you need to do a boolean comparison, you use assert_true. If you don't know your input type, you should probably validate/convert that first. If you don't need to do a boolean comparison (i.e. if you wouldn't want "Alibaba" to evaluate to True), then you just don't use the boolean assertion?

The point of these functions is to check how something would be evaluated to a boolean, as in the example I gave where the program flow is based on if(result). We don't seem to have that need in our test suite atm, so hence we shouldn't have assert_true.

Copy link
Author

@vincenzopalazzo vincenzopalazzo Aug 2, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mh! I do not argue this on the PR because it is the wrong place but I disagree with this statement

How is that worse? You just need to use the right tool for the job.

Of course, bug is a bug, and it is there because there is a mistake in the code, so if a comparison between bool, I want to compare boolean value and not a string that always evaluates to true except for empty.

Is worse because if we were able to catch this bug easily we were still coding in C without struggling with rust compiler, if you do this case you should define some rules like: bool("False") -> False or True?

Imagine to have assert_true(json['foo']) and make an API change that move the type of a filed from string to int

you fall in case of

Python 3.8.10 (default, Jun 22 2022, 20:18:18) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> bool(0)
False
>>> bool("0")
True
>>> bool(0.0)
False
>>> bool("0.0")
True

It is not the right tool anyway at least without a good reason to allow this, and I do not see any till now

Copy link
Author

@vincenzopalazzo vincenzopalazzo Aug 2, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A better example can be

take the from our assert_true(hash_str is None) with assert_true(hash_str)

➜  ~ python3
Python 3.8.10 (default, Jun 22 2022, 20:18:18) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> bool(None)
False
>>> bool("")
False

This can be a bug not catch by the CI, and if it is an API change you can break the client that use a strongly typed language

@vincenzopalazzo
Copy link
Author

vincenzopalazzo commented Aug 2, 2022

In this PR I have no strong opinion in whatever decision we take, and I just do not care too much about what type of assert we will use, as I see this will not bring too much value inside the code base.

I'm fine with assert_is_none(x), but it is possible to have the same result with assert_true(x is None, err_msg=f"wrong value: {x}") or assert_equal(x, None, err_msg=f"wrong value {x}")

more function we add, more code to maintain we have, and if there is no strong agreement I will preferer to remove all the asser_* functions and keep just assert_equal because if we add other functions here we also need to migrate the old code, and migrate test unit IMHO it is never a good idea

@stickies-v
Copy link
Contributor

Yeah okay let's do the common denominator approach now and potentially add more functions later on, I don't want to push on the is thing too much. So I think in short, remove assert_true() in favour of generic assert statements, and use assert_equal for the other uses? I'll re-review once that's updated.

@NorrinRadd
Copy link

Dup from IRC: this PR has been quiet for 2 months. Is it ok if someone else completes the last recommendations?

Secondly, the latest ask is essentially the opposite of the the ask in the issue this PR is meant to address. Can we get some confirmation on what the approach should be? cc: @theStack

@vincenzopalazzo
Copy link
Author

Dup from IRC: this PR has been quiet for 2 months. Is it ok if someone else completes the last recommendations?

Feel free to cherry-pick my commits and continue in a new PR it is totally fine, and also it useful that someone else will continue an unfinished work

@fanquake
Copy link
Member

fanquake commented Oct 3, 2022

Feel free to cherry-pick my commits and continue in a new PR it is totally fine,

@vincenzopalazzo does that mean you're no-longer working on / maintaining this PR? If so, would you like to close it?

@vincenzopalazzo
Copy link
Author

@fanquake Yes I think that there are people that can give a better love to this PR

@bitcoin bitcoin locked and limited conversation to collaborators Oct 3, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants