Skip to content

Commit 9847b09

Browse files
wesbillmanPinky
andauthored
test(desktop): isolate login-shell probe measurements (#7570)
Pinky, acting on Wes’s behalf. ## Summary Fix a pre-existing false-failure mechanism in the login-shell discovery tests, independently of the thread-window work. - Run the two process-global-counter assertions in separate, bounded libtest processes. The global counter still sees all of each test’s auth/worker threads; no production discovery or cache behavior changes. - Require exact test selection, successful child exit, and a completion receipt after the assertion body. Reject counter resets outside isolation. - Keep the cold/repeated cheap-discovery zero assertions and forced-discovery positive control unchanged. Add parent-probe exclusion and own-worker controls. Three test-only files, +103/-4. Base `00209076c7a10d9e4a475466c313e8ebecf041f5`; head `07f8e86c58a0aa5378537a3daa4610361fdbad55`. ## Reproduction and causal checks On the main-based source, cheap discovery alone counted 0. Running the existing missing-binary version test on another thread inside the measurement interval made the repeated-cheap assertion fail with 1. This reproduces interference, not the exact scheduling/caller behind the original full-suite failure. The counter measures `run_in_login_shell` invocations, not exact OS process counts. Mutation checks: - Removing isolation fails the measurement guard. - Removing the real production `record()` call fails the original forced positive control (0). - A reviewer-identified early-return mutation initially survived; the final helper owns the assertion closure instead. Deleting that invocation now fails the external worker assertion (0 vs 1). Independent read-only review by Brain found no remaining blocking finding at the final head. ## Validation at the final head - **Full `just ci`: passed.** Desktop Tauri: 3,175 passed, 19 ignored; desktop JS: 6,488 passed; mobile: 2,098 passed. Remaining Rust unit, formatting/lint/static, and desktop/web build lanes passed. An initial five-minute tool timeout interrupted compilation; the subsequent complete run used a longer deadline, not altered gates. - **Normal pre-commit and pre-push hooks: passed.** Push includes default + mesh Tauri Clippy, complete Tauri workspace tests, size and branch-scope checks. No bypass. - **Real local relay, following `TESTING.md`: passed.** Built release `buzz-relay`, `buzz`, and `buzz-admin` from this head. Fresh isolated Postgres/Redis, additive migrations, dedicated loopback ports, disposable NIP-98 identity; health, create channel, signed root/reply send, exact-ID get, and thread retrieval passed. Relay and both owned containers stopped afterward. Dev auth-token mode follows the guide; not a production-auth or thread-window/native UI acceptance claim. ## Scope and limits This is process/cache isolation, not a filesystem sandbox. The existing bounded-command helper bounds child runtime/output; its Unix process-group behavior does not guarantee ownership of nested probe groups. Existing individual probe deadlines remain unchanged. No UI change, screenshots, thread-window commits, feature enablement, merge, or deployment included. Originating conversation: buzz://message?channel=0b73400d-6922-4fe3-9b97-454da04d011b&id=984528c883faa8829971d19394870f0f2d543d5c1950565fd55ffddc1b8d8022 Signed-off-by: Pinky <5f5ab050ec58ae208332edd544ebf705221e24c1b86d82a6ca07038a7a8f6ac9@buzz.block.builderlab.xyz> Co-authored-by: Pinky <5f5ab050ec58ae208332edd544ebf705221e24c1b86d82a6ca07038a7a8f6ac9@buzz.block.builderlab.xyz>
1 parent 6146c4f commit 9847b09

3 files changed

Lines changed: 103 additions & 4 deletions

File tree

‎desktop/src-tauri/src/managed_agents/discovery/login_shell_spawn_probe.rs‎

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,9 +13,96 @@ pub(crate) fn record() {
1313
}
1414

1515
pub(crate) fn reset() {
16+
#[cfg(unix)]
17+
assert!(
18+
is_isolated_process(),
19+
"counter measurements must run in an isolated test process"
20+
);
1621
COUNT.store(0, Ordering::SeqCst);
1722
}
1823

1924
pub(crate) fn count() -> usize {
2025
COUNT.load(Ordering::SeqCst)
2126
}
27+
28+
#[cfg(unix)]
29+
const CHILD_TEST: &str = "BUZZ_LOGIN_SHELL_PROBE_TEST";
30+
31+
#[cfg(unix)]
32+
fn is_isolated_process() -> bool {
33+
std::thread::current()
34+
.name()
35+
.is_some_and(|name| std::env::var(CHILD_TEST).as_deref() == Ok(name))
36+
}
37+
38+
/// Run a counter-owning test body alone in a fresh libtest process.
39+
///
40+
/// The counter deliberately stays process-wide: discovery's auth workers must
41+
/// count too. A voluntary PATH lock cannot exclude unrelated, unlocked probe
42+
/// callers in the full suite. Process isolation excludes those callers without
43+
/// changing production code or requiring new workers to inherit test context.
44+
/// Call while holding the PATH lock so the child inherits stable environment.
45+
#[cfg(unix)]
46+
pub(crate) fn run_in_isolated_process(test: impl FnOnce()) {
47+
use std::process::Command;
48+
use std::time::Duration;
49+
50+
let thread = std::thread::current();
51+
let name = thread.name().expect("libtest names its test threads");
52+
let completed = format!("completed isolated login-shell probe test: {name}");
53+
if is_isolated_process() {
54+
test();
55+
// Receipt only after the assertion body returns, never on entry.
56+
println!("{completed}");
57+
return;
58+
}
59+
60+
let mut command = Command::new(std::env::current_exe().expect("test executable"));
61+
command
62+
.args(["--exact", name, "--nocapture", "--test-threads=1"])
63+
.env(CHILD_TEST, name);
64+
let output = super::bounded_command::output_with_timeout(command, Duration::from_secs(300))
65+
.expect("isolated probe test must finish within five minutes and the output cap");
66+
let stdout = String::from_utf8_lossy(&output.stdout);
67+
let stderr = String::from_utf8_lossy(&output.stderr);
68+
assert!(
69+
output.status.success() && stdout.lines().any(|line| line.contains(&completed)),
70+
"isolated probe test {name} failed or did not run: {}\n{stdout}\n{stderr}",
71+
output.status
72+
);
73+
}
74+
75+
#[cfg(unix)]
76+
#[test]
77+
fn isolated_counter_excludes_parent_probes_but_counts_own_workers() {
78+
let _guard = crate::managed_agents::lock_path_mutex();
79+
let probe_on_worker = || {
80+
std::thread::spawn(|| {
81+
// A real production probe, independent of the shared PATH cache.
82+
super::find_via_login_shell("buzz-absent-probe-isolation-xyzzy")
83+
})
84+
.join()
85+
.expect("probe worker must finish")
86+
};
87+
if !is_isolated_process() {
88+
assert!(probe_on_worker().is_none());
89+
assert!(count() >= 1, "parent worker must reach the real probe");
90+
}
91+
run_in_isolated_process(|| {
92+
// Before reset: unrelated parent probes must not enter this process.
93+
assert_eq!(count(), 0, "the child must start with its own counter");
94+
reset();
95+
assert!(probe_on_worker().is_none());
96+
});
97+
if is_isolated_process() {
98+
// Outside the closure so omitting its invocation cannot pass this test.
99+
assert_eq!(count(), 1, "the child's assertion body and worker must run");
100+
}
101+
}
102+
103+
#[cfg(unix)]
104+
#[test]
105+
#[should_panic(expected = "counter measurements must run in an isolated test process")]
106+
fn reset_rejects_unisolated_measurement() {
107+
reset();
108+
}

‎desktop/src-tauri/src/managed_agents/discovery/tests/forced_discovery.rs‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -108,14 +108,21 @@ fn forced_discovery_probes_auth_but_cheap_discovery_reuses_cached_status() {
108108
#[cfg(unix)]
109109
#[test]
110110
fn cheap_discovery_reports_absent_before_any_forced_probe() {
111+
let _path_guard = crate::managed_agents::lock_path_mutex();
112+
super::super::login_shell_spawn_probe::run_in_isolated_process(
113+
cheap_discovery_reports_absent_before_any_forced_probe_body,
114+
);
115+
}
116+
117+
#[cfg(unix)]
118+
fn cheap_discovery_reports_absent_before_any_forced_probe_body() {
111119
use crate::managed_agents::custom_harnesses::registry_test_lock;
112120
use crate::managed_agents::discovery::{
113121
clear_resolve_cache, discover_acp_runtimes_from, login_shell_spawn_probe,
114122
};
115123
use crate::managed_agents::{AcpAvailabilityStatus, AuthStatus};
116124
use std::os::unix::fs::PermissionsExt;
117125

118-
let _path_guard = crate::managed_agents::lock_path_mutex();
119126
let _registry_guard = registry_test_lock();
120127

121128
let dir = tempfile::tempdir().expect("tempdir");

‎desktop/src-tauri/src/managed_agents/discovery/tests/managed_path_resolution.rs‎

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -126,16 +126,21 @@ fn resolve_command_prefers_buzz_managed_npm_shim_over_path() {
126126
#[cfg(unix)]
127127
#[test]
128128
fn cheap_discovery_never_spawns_login_shell_even_when_cold() {
129+
let _path_guard = crate::managed_agents::lock_path_mutex();
130+
super::super::login_shell_spawn_probe::run_in_isolated_process(
131+
cheap_discovery_never_spawns_login_shell_even_when_cold_body,
132+
);
133+
}
134+
135+
#[cfg(unix)]
136+
fn cheap_discovery_never_spawns_login_shell_even_when_cold_body() {
129137
use crate::managed_agents::custom_harnesses::registry_test_lock;
130138
use crate::managed_agents::discovery::{
131139
clear_resolve_cache, discover_acp_runtimes_from, login_shell_spawn_probe,
132140
};
133141
use std::fs;
134142
use tempfile::tempdir;
135143

136-
// Serialize with every other test that spawns a login shell: the spawn
137-
// counter and the PATH/login-shell caches are process-global.
138-
let _path_guard = crate::managed_agents::lock_path_mutex();
139144
let _registry = registry_test_lock();
140145

141146
// A custom harness whose command cannot resolve anywhere, so the resolver

0 commit comments

Comments
 (0)