Skip to content

Conversation

@Empact
Copy link
Contributor

@Empact Empact commented May 22, 2018

Files should depend on one another by interface, not by implementation.
This checks for quoted includes as well.

@Empact Empact force-pushed the lint-include-cpp branch from ae69816 to 177dc42 Compare May 22, 2018 03:24
@Empact
Copy link
Contributor Author

Empact commented May 22, 2018

With @practicalswift #13288 (comment)

Pending #13288, #13291

Current failures, via travis:

$ if [ "$TRAVIS_EVENT_TYPE" = "pull_request" ]; then contrib/devtools/lint-all.sh; fi
The following files #include .cpp files:
src/test/blockchain_tests.cpp:#include "rpc/blockchain.cpp"
src/test/torcontrol_tests.cpp:#include <torcontrol.cpp>
^---- failure generated from contrib/devtools/lint-include-cpp.sh

@fanquake fanquake added the Tests label May 22, 2018
@ken2812221
Copy link
Contributor

Concept ACK

@practicalswift
Copy link
Contributor

practicalswift commented May 22, 2018

ACK 177dc4277722c2e3d5c34bab4de3951055510a88

Very nice! :-) Love these small regression tests that make sure we'll never have to think about a given class of otherwise recurring mistakes/issues ever again.

Tiny nit: Could be added to existing lint-includes.sh instead if we want to keep the number of linter files down.

Copy link
Contributor

@promag promag left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit

#!/usr/bin/env bash

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, every bash invocation is via #!/bin/bash. I'm seeing arguments for using bin/env for compatibility across more systems, e.g. OpenBSD, and some concerns about potential security issues if malicious code corrupts the path. Not sure how to value those.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these glob portable?

Copy link
Contributor

@practicalswift practicalswift May 22, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@promag They are expanded by git and not the shell, but even if they were handled by bash they would have be portable, no? :-)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@promag They are expanded by git

👍

but even if they were handled by bash they would have be portable, no?

See https://github.com/bitcoin/bitcoin/pull/13228/files#r188147943.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Provided it's run from the project root (which is how it's run on Travis), it should get all cpp/h files - it's just the top level that is missed by ** in some cases.

Copy link
Contributor

@practicalswift practicalswift May 23, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I now understand what you meant. What about changing it from "**/*.cpp" "**/*.h" to "*.cpp" "*.h"? That will work regardless of where it is run from.

@Empact Empact force-pushed the lint-include-cpp branch from 177dc42 to 0636867 Compare May 23, 2018 05:30
@laanwj
Copy link
Member

laanwj commented May 23, 2018

Sorry to be skeptical here, but does this really happen enough to merit adding yet another linter?

@Empact
Copy link
Contributor Author

Empact commented May 23, 2018

@laanwj Generally, I'm in favor of more lints as implementing practice in code means that it need not be propagated / enforced by humans - particularly when the difference is easily overlooked by humans and easily detected by code. This case has been violated twice, and overlooked twice by reviewers, so an automatic check seems helpful, if we agree that it should not be violated.

On that point, I agree with MarcoFalke: "Including a cpp file seems extremely wrong."
#13230 (comment)

That said, I'm not clear how to value the cost of having an extra file in the contrib/devtools directory, or having this command run with each CI run.

@maflcko
Copy link
Member

maflcko commented May 23, 2018

Can this be combined with the other include linter, so we don't have too many "duplicate" files?

@Empact Empact force-pushed the lint-include-cpp branch from 0636867 to 2f2a784 Compare May 23, 2018 21:31
@promag
Copy link
Contributor

promag commented May 23, 2018

Can this be combined with the other include linter, so we don't have too many "duplicate" files?

I wouldn't mind separate files as it would allow:

find contrib/devtools -name "lint-*.sh" ! -name lint-all.sh | parallel -u bash :::

However flake8 takes most of the time so no gain for now (hint: s/bash/time above).

@promag
Copy link
Contributor

promag commented May 23, 2018

Should it just check if the included file is header|header.h|header.hpp?

@practicalswift
Copy link
Contributor

ACK 2f2a784c7ba189628dfc5d9dbd10d1861ebaadbd

@laanwj
Copy link
Member

laanwj commented May 24, 2018

Generally, I'm in favor of more lints as implementing practice in code means that it need not be propagated / enforced by humans

I agree. I don't think I have a problem with it in principle.
It's just the proliferation of all kinds of regexp shell scripts that might make the travis build fail, that I'm having problems with. There is a cost to this in maintenance.

I think linters should be used to avoid sneaky problems that might result in bugs. For example one I really like is #13041, which catches potential locale-dependency bugs.

Including a .cpp file is like including a 10000-pound elephant in the room, if reviewers don't catch this we have a really big problem.

@Empact
Copy link
Contributor Author

Empact commented May 24, 2018

Agreed, #13041 is a great one. I mean to cite the fact that this has happened before as evidence that these are tricky, perhaps because it is easy to gloss over includes, particularly in the context of tests. For reference, the PRs where these includes were introduced: #10408 #11748

@practicalswift
Copy link
Contributor

practicalswift commented May 24, 2018

@laanwj Adding some data: there seems to have been a total of twelve .cpp includes during the project git history:

$ git log -u | grep -E '^\+#include.*\.cpp.$' | cut -b2- | tr '<>' '"' | sort -u
#include "base58_tests.cpp"
#include "base64_tests.cpp"
#include "Checkpoints_tests.cpp"
#include "DoS_tests.cpp"
#include "miner_tests.cpp"
#include "rpc/blockchain.cpp"
#include "script_tests.cpp"
#include "torcontrol.cpp"
#include "transaction_tests.cpp"
#include "uint160_tests.cpp"
#include "uint256_tests.cpp"
#include "util_tests.cpp"

This issue has been independently introduced twice in the code base in two separate PR:s during the last twelve months alone. The PRs (#10408, #11748) were reviewed by very skilled developers before merge.

Thus it empirically seems like an extra pair of very attentive eyes in the form of Travis would be helpful in making sure this 10000-pound elephant doesn't sneak into the code base again :-)

@practicalswift
Copy link
Contributor

@laanwj Regarding the potential maintenance need for the linting shell scripts: I volunteer to maintain all the linter scripts in the repo if that is of any help. In the event of a linter causing problems just ping me in and I'll fix it quickly! :-)

@laanwj
Copy link
Member

laanwj commented May 24, 2018

@laanwj Adding some data: there seems to have been a total of twelve .cpp includes during the project git history:

Even on the desirability there is an alternative perspective, FWIW: at one of my previous employers we maintained a huge system, and including the .c or .cpp was standard practice for private unit tests. This allowed testing internal functions and methods and which are not exposed on the public interface of a compilation unit.
(an alternative to this at a slightly higher level was to have a "test interface", but this carries a bit of run time overhead)

Not saying that we should do that here, but it's all too easy to be blanket against something if it offends your aesthetic sensibilities.

Thus it empirically seems like an extra pair of very attentive eyes in the form of Travis would be helpful in making sure this 10000-pound elephant doesn't sneak into the code base again :-)

Heh :)

@laanwj Regarding the potential maintenance need for the linting shell scripts: I volunteer to maintain all the linter scripts in the repo if that is of any help. In the event of a linter causing problems just ping me in and I'll fix it quickly! :-)

Okay! Thanks.

@maflcko
Copy link
Member

maflcko commented Jun 5, 2018

I wouldn't mind separate files as it would allow:
find contrib/devtools -name "lint-*.sh" ! -name lint-all.sh | parallel -u bash :::
However flake8 takes most of the time so no gain for now (hint: s/bash/time above).

I highly doubt that a single git grep makes up a significant run-time to justify for the overhead of another script with copyright header, etc.

@maflcko
Copy link
Member

maflcko commented Jun 5, 2018

Oh, I see this was already combined into the existing file. utACK, but needs rebase.

Files should depend on one another by interface, not by implementation.
This checks for quoted includes as well.

With practicalswift
@Empact Empact force-pushed the lint-include-cpp branch from 2f2a784 to 9d6c9db Compare June 6, 2018 09:11
Copy link
Contributor

@ken2812221 ken2812221 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK 9d6c9db

@Empact
Copy link
Contributor Author

Empact commented Jun 6, 2018

Rebased for #13385

@maflcko maflcko merged commit 9d6c9db into bitcoin:master Jun 6, 2018
maflcko pushed a commit that referenced this pull request Jun 6, 2018
9d6c9db lint: Add linter to error on #include <*.cpp> (Ben Woosley)

Pull request description:

  Files should depend on one another by interface, not by implementation.
  This checks for quoted includes as well.

Tree-SHA512: d36d468f48d538077f5f927b9561729fd7d76319f6b2e2cc10414a9f243588194e90ca1d85eca65019f9259268f555d25106eaaa56da28c58fa8d5837b469661
@Empact Empact deleted the lint-include-cpp branch July 2, 2018 02:59
PastaPastaPasta pushed a commit to PastaPastaPasta/dash that referenced this pull request Jul 2, 2020
9d6c9db lint: Add linter to error on #include <*.cpp> (Ben Woosley)

Pull request description:

  Files should depend on one another by interface, not by implementation.
  This checks for quoted includes as well.

Tree-SHA512: d36d468f48d538077f5f927b9561729fd7d76319f6b2e2cc10414a9f243588194e90ca1d85eca65019f9259268f555d25106eaaa56da28c58fa8d5837b469661
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants