Skip to content

fix(tui): give cargo the absolute path of protoc, so it stops rebuilding pi-tools-api - #30

Merged
zy1233 merged 4 commits into
ci/release-setup-protoc-tokenfrom
fix/tui-protoc-absolute-path
Oct 20, 2026
Merged

zy1233 merged 4 commits into
ci/release-setup-protoc-tokenfrom
fix/tui-protoc-absolute-path

Conversation

@zy1233

@zy1233 zy1233 commented Oct 20, 2026 •

Copy link
Copy Markdown
Owner

Summary

Plan item protoc_abs, chosen by the owner on 2026-10-20: find_protoc() returns the absolute path of protoc when it finds the tool on PATH. Before, it returned the bare name.

build script of pi-tools-api
  pi_proto_build::compile_protos
    find_protoc()
      1. $PROTOC                    exists?  -> that path
      2. bin/protoc (walk up)       runs?    -> relative path  (exists, so cargo can track it)
      3. protoc on PATH
-        check "protoc"                       -> "protoc"       (no such file in the package dir)
+        for each dir in PATH: <dir>/protoc, a file with an execute bit, runs?
+                                             -> /abs/path/protoc
    emit_rerun_if_changed
      println!("cargo:rerun-if-changed={protoc}")   <- cargo: missing path = changed in every run

cargo:rerun-if-changed=protoc names a path that does not exist in the package directory. Cargo treats such a path as changed in every run (its own words: stale: missing ".../pi-tools-api/protoc"). So each cargo run started the build script again and rebuilt pi-tools-api and the crates above it. CI has protoc on PATH (arduino/setup-protoc) and no PROTOC variable, so CI hit this. A developer with an absolute PROTOC did not. How much CI gains is small (see "CI result, and a correction"): the loop shows only when the same cargo command runs again. Only one build script uses pi-proto-build (pi-tools-api/build.rs).

crates/build/pi-proto-build/src/find_protoc.rs
├── executables_on_path(name, path, cwd)   NEW  PATH search; absolute results; file + execute bit
├── is_executable_file                     NEW  (unix: mode & 0o111; else is_file)
├── find_protoc  step 3                    uses it; the first candidate that runs wins
└── tests                                  8 new (7 for the search, 1 regression test)
CHANGELOG.md                               1 entry (Fixed)

Base: ci/release-setup-protoc-token (#29), the same chain as #28. Not merged. #27, #28 and #29 are not merged either.

One difference to know: for a protoc found on PATH, find_protoc_include_dir now gets an absolute path, so pi-proto-build also passes -I<prefix>/include (the directory next to bin/, if it exists). It already did this for an absolute PROTOC. On CI the directory should exist (setup-protoc unpacks bin/ and include/), and the TUI CI run of this PR builds the protos without error. The log does not print the protoc arguments, so it does not show that the flag was passed. On the WSL machine used below the directory does not exist, so that flag was not tested there.

Evidence

WSL2, protoc 28.3 on PATH, PROTOC unset, build output in tmpfs, cargo check --locked -p pi-tools-api, nothing changed between the runs:

Run Before (parent commit) After (this PR)
2 pi-tools-api is stale, rebuilt (2 s) not rebuilt (1 s)
3 Compiling pi-tools-api ... (2.2 s) Finished only (0.33 s)
  • Before: cargo's fingerprint log says stale: missing "<repo>/tui/crates/codegen/pi-tools-api/protoc".
  • After: the third run prints only Finished, and the fingerprint log has no line for pi-tools-api.
  • The file that the build script generates (pi.grok.tools.v1.rs) has the same sha256 before and after.
  • cargo test -p pi-proto-build --lib: 10 passed (the 8 new and the 2 old tests). cargo fmt --check and cargo clippy --all-targets for the package: clean.
  • Mutation check: the old find_protoc.rs with only the new regression test fails:
test find_protoc::tests::the_path_that_find_protoc_gives_is_a_file_that_exists ... FAILED
`protoc` is not a file

CI result, and a correction

TUI CI (run 39356795564) and CI (run 39356795545) passed on 7a4ce9d: 8,490 passed / 0 failed / 21 ignored, and every suite equals its reference number. The commits after 7a4ce9d change docs only.

The gain in CI is small:

Before After
Gates + cargo check job: compiles of pi-tools-api 2 1
Gates + cargo check job: time 3 min 19 s to 4 min 27 s 3 min 17 s
cargo test job: compiles of pi-tools-api / pi-shell / pi-pager 7 / 4 / 3 7 / 4 / 3
cargo test job: time 22 min 04 s to 22 min 43 s 22 min 08 s

"Before": the compile counts come from the run 39193752451; the times come from the three runs for #28.

The cargo check job runs two cargo commands with the same packages, so it showed the loop: one compile less now. The cargo test job runs one cargo command for each suite, and each command builds the dependencies with the features of its own packages. So pi-tools-api is compiled once in each of the 7 suites that use it, with or without this PR, and the same suites compile pi-shell and pi-pager before and after.

An earlier version of this description, the CHANGELOG entry and the execution log said that the 7 / 4 / 3 compiles were the loop. That was wrong, and the CI run of this PR shows it. The CHANGELOG entry and the execution log are corrected in e2de4c4. The effect of this PR is on a developer's machine, where the same command runs again and again.

Merge Danger

Door: two-way. One function in a build helper; revert it at any time.

Blast Radius: the build of pi-tools-api (the only user of pi-proto-build), and with it every crate above it, on Linux and macOS. If protoc were in a place that the old check found and the new search does not, the build would fail at that crate. I know of no such place: on Linux and macOS, Command::new looks in the PATH directories only, as the new search does. The TUI does not build on Windows.

find_protoc() returned the bare name `protoc` when it found the tool on
PATH. The build script of pi-tools-api printed it as
`cargo:rerun-if-changed=protoc`. That path is no file in the package
directory, and cargo treats a path that does not exist as changed in
every run. So every cargo run ran the build script again and rebuilt
pi-tools-api. CI has protoc on PATH (arduino/setup-protoc) and no
PROTOC variable, so it showed this: pi-tools-api was compiled 7 times
in the cargo test job.

find_protoc() now looks for the file in the directories of PATH and
returns its absolute path. An absolute PROTOC variable already worked.
The item leaves the list of open decisions. It is done in PR 30.
The CHANGELOG entry and the section 10.20 paragraph said that the 7 / 4 / 3
compiles of pi-tools-api / pi-shell / pi-pager in the `cargo test` job came
from the rerun-if-changed loop. The CI run of this PR shows that they do not:
the numbers are the same after the fix, because the job runs one `cargo`
command for each suite and each command builds the dependencies with its own
features. Only the `cargo check` job shows the loop (2 compiles of
pi-tools-api before, 1 after). The job times did not change (22 min 08 s,
against 22 min 04 s to 22 min 43 s before).

Say this in both places, and write down the result of the CI run
(8,490 passed / 0 failed / 21 ignored). Correct the sentence about the
benefit in the plan, too.
@zy1233

zy1233 commented Oct 20, 2026

Copy link
Copy Markdown
Owner Author

TUI CI (run 39356795564) and CI (run 39356795545) passed on 7a4ce9d: 8,490 passed / 0 failed / 21 ignored; every suite equals its reference number.

I corrected the description (section "CI result, and a correction"), the CHANGELOG entry and the execution log (e2de4c4). I had written that the 7 / 4 / 3 compiles of pi-tools-api / pi-shell / pi-pager in the cargo test job were the rebuild loop, and that this PR would remove them. The CI run shows that they are not: the numbers are the same after the fix, because that job runs one cargo command for each suite and each command builds the dependencies with its own features. The loop shows in the cargo check job (2 compiles of pi-tools-api before, 1 after) and on a developer's machine, where the same command runs again and again. The job time did not change (22 min 08 s; 22 min 04 s to 22 min 43 s in the runs for #28).

The -I<prefix>/include flag: the build passes on CI, but the log does not print the protoc arguments, so it does not show that the flag was passed.

zy1233 added a commit that referenced this pull request Oct 20, 2026
- Plan (status, section 10 index, revision log), execution log (new section
  10.21), CHANGELOG, baselines and baseline history: the numbers of the first
  nextest run on the whole tree (TUI CI run 39359879537: 8,490 passed, 0 failed,
  22 skipped), the test time (about 130 s with nextest, about 65 s with
  cargo test, same code), and the dry run of release.yml for PR #29.
- Correct the r17 notes: the 7 / 4 / 3 compiles of pi-tools-api in the cargo
  test job were not caused by the protoc path (PR #30 shows it), and the runner
  never ran doctests, so nextest loses nothing there.
- Remove a duplicated "verification" heading from the r17 text.
@zy1233
zy1233 added this pull request to stack #32 October 20, 2026 06:16
@zy1233
zy1233 merged commit d2182d9 into main Oct 20, 2026
7 checks passed
@zy1233
zy1233 deleted the fix/tui-protoc-absolute-path branch October 20, 2026 06:19
zy1233 added a commit that referenced this pull request Oct 20, 2026
* ci(tui): run the TUI tests with cargo nextest

The vendored tests were written for cargo nextest (one process per test), and some say in a
comment that they are reliable only under it. TUI CI ran cargo test, where all tests of a
binary share one process.

- scripts/tui_baseline.toml: `[test] runner = "nextest"`; `--runner {cargo,nextest}` on the
  command line takes the other one. The libtest path stays as it was.
- scripts/tui_baseline.py: `parse_nextest_output` reads the Summary line (nextest counts the
  ignored tests and the tests removed by --skip together as "skipped") and the recap of the
  failed tests (the live part has the captured output of a test, so it is not read);
  `suite_command` builds the command line; the report names the runner; a missing
  cargo-nextest stops the run at once, with a hint.
- stream(): after the timeout, SIGTERM to the process group, and SIGKILL 20 s later. nextest
  puts each test in a process group of its own, so a SIGKILL to nextest left a hanging test
  alive (measured on a demo crate); a nextest that gets a SIGTERM stops its tests.
- tui-ci.yml: install cargo-nextest 0.9.148 (taiki-e/install-action) in the test job; the job
  is now called "cargo nextest (Linux x86_64)".
- 21 new unit tests. The fixtures are real output of nextest 0.9.148.

* docs(tui): say in AGENTS.md and the tui_envspy README that TUI CI runs nextest

* docs: record the nextest change and the r18 round

- Plan (status, section 10 index, revision log), execution log (new section
  10.21), CHANGELOG, baselines and baseline history: the numbers of the first
  nextest run on the whole tree (TUI CI run 39359879537: 8,490 passed, 0 failed,
  22 skipped), the test time (about 130 s with nextest, about 65 s with
  cargo test, same code), and the dry run of release.yml for PR #29.
- Correct the r17 notes: the 7 / 4 / 3 compiles of pi-tools-api in the cargo
  test job were not caused by the protoc path (PR #30 shows it), and the runner
  never ran doctests, so nextest loses nothing there.
- Remove a duplicated "verification" heading from the r17 text.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant