|
| 1 | +# Tooling Overview |
| 2 | + |
| 3 | +This document explains the high-level structure and philosophy of this |
| 4 | +repository tooling. It is intended for developers who want to contribute to |
| 5 | +the tooling. |
| 6 | + |
| 7 | +## Guiding Principles |
| 8 | + |
| 9 | +* **Dart tooling is better than shell scripts.** Non-trivial tooling should be |
| 10 | + written and maintained in the language the team is most familiar with. Dart |
| 11 | + is also safer and more testable than shell scripts. |
| 12 | +* **Everything should be runnable locally.** It should be trivial to run the |
| 13 | + same tests locally as are run in CI. This makes the development cycle much |
| 14 | + simpler, as contributors don't need to rely on CI cycles to get feedback, and |
| 15 | + also makes investigating failures faster and easier. |
| 16 | +* **Auditability is critical.** It should be trivial to determine which tests |
| 17 | + ran on which packages, and what the outcome of each was. When they don't run |
| 18 | + for a given package, it should be clear that they didn't, and why. |
| 19 | +* **False negatives are the worst kind of failure.** In general, it's much |
| 20 | + better for the default to be to fail than to silently pass. If most packages |
| 21 | + should run a certain kind of test, then not having that test without explicit |
| 22 | + opt-out should fail. |
| 23 | +* **Tooling is better at enforcing things than humans.** If a repo standard or |
| 24 | + best practice can be encoded in the tooling, it should be. Reviewers should |
| 25 | + not need to remember to actively check for every possible violation. When |
| 26 | + there's judgment involved, consider defaulting to failing, and having an |
| 27 | + `override: *` label to allow reviewers to intentionally skip a check in |
| 28 | + presubmit. |
| 29 | +* **Collecting failures is better than stopping immediately.** Unless a failure |
| 30 | + would prevent continuing, it is better to add the failure to a list and |
| 31 | + continue, then report all failures. If a package has N unrelated failures, |
| 32 | + it's much better to report all of them in one run, rather than requiring N |
| 33 | + fix/re-run cycles. |
| 34 | + |
| 35 | +## Structure |
| 36 | + |
| 37 | +* Most commands should be subclasses of `PackageLoopingCommand`, which handles |
| 38 | + all of the logic of iterating over packages and summarizing the results. |
| 39 | + Unless a failure is systemic, failures should be returned rather than thrown |
| 40 | + so that the command can continue to run for the remaining packages. |
| 41 | + * Overriding `packageLoopingType` allows controlling which packages are |
| 42 | + included (e.g., top-level only vs all sub-packages). |
| 43 | + * Overriding `initializeRun()` allows for setup that should happen once, such |
| 44 | + as reading/parsing repo-level files needed for all packages. |
| 45 | + * Overriding `completeRun()` allows for final one-time checks based on the |
| 46 | + results of all package runs. |
| 47 | + * Overriding `shouldIgnoreFile()` allows for skipping commands when the only |
| 48 | + files touched cannot possibly impact the result of the command (e.g., |
| 49 | + `README.md` changes for a native unit test command). `file_filters.dart` has |
| 50 | + several common classes of files that can be useful for filtering. |
| 51 | + * Keep in mind the principle that false negatives are the worst kind of |
| 52 | + failure, so err on the side of *not* skipping unless you're very sure |
| 53 | + it's safe. |
| 54 | +* Commands that are not package-level (e.g., the license check should run on |
| 55 | + every file in the repository, whether part of a package or not) should instead |
| 56 | + subclass the more generic `PackageCommand`. |
| 57 | +* Very fast and simple checks should be new classes in `lib/src/validators/` |
| 58 | + rather than new commands, and added to `ValidateCommand`. While this is |
| 59 | + slightly worse for looking at failures in CI, it is much easier for |
| 60 | + contributors (and their agents) to run one command to do most of the checks |
| 61 | + rather than remembering the entire suite of commands to run. |
| 62 | + |
| 63 | +## Types of Commands |
| 64 | + |
| 65 | +Commands generally fall into two categories: checks and developer utilities. |
| 66 | +* Checks are intended to be run by CI as well as locally. They generally |
| 67 | + validate that some desired condition is met (unit tests pass, pubspecs are |
| 68 | + well structured, etc.) and have a pass/fail result. |
| 69 | +* Developer utilities are not intended to be used in CI, but instead to make |
| 70 | + things easier for developers. These should be documented in `README.md` for |
| 71 | + easier discovery. |
| 72 | + |
| 73 | +There is some overlap between these two categories. For example, `format` |
| 74 | +is primarily a developer utility, but with the `--fail-on-change` flag |
| 75 | +can be used as a check in CI. |
0 commit comments