Repository navigation
Commit d37a9f4
authored
fix(apple-runner): restart without resending a command written then lost (#3082)
* fix(apple-runner): name the lost reply when status cannot place a lost command
A command whose reply was lost and whose status probe answers notAccepted
(a restarted runner's empty journal) or an unnamed state now fails with
details.reason runner_reply_lost beside dispatched unknown, so a caller
keys on the typed reason instead of the message before it observes the
screen.
Tests (fake runner server through runAppleRunnerCommand and the recovery
module):
- runner-recovery-wiring: press and fill (runner tap and type) accept the
command, drop the connection, status answers notAccepted: one dispatched
command, error dispatched unknown with reason runner_reply_lost, the
session is invalidated, and the next command gets a runner. Mutation:
delete `reason: RUNNER_REPLY_LOST_REASON` from the unknown-state error ->
both rows fail.
- runner-recovery-wiring: a snapshot whose reply is dropped is resent and
succeeds. Mutation: declare snapshot with DEFAULT_TRAITS (not read-only)
-> fails (the lost read goes to status recovery and is not resent).
- runner-command-recovery: notAccepted yields reason runner_reply_lost and
dispatched unknown. Mutation: same deletion as above -> fails.
* fix(apple-runner): restart without resending a command written then lost
A connect-shaped failure keeps its restart, but the command is sent again
on the restarted runner only with proof the first send could not have run
it: the attempt provably wrote nothing (a connect refused before the write,
dispatched no), the readiness preflight gave up before the write, or the
command is read-only by its runner trait. A mutating command whose first
attempt may have written it now fails with reason runner_reply_lost and
dispatched unknown after the restart; a restarted runner's journal is empty,
so no status answer can prove the first send did not run. A read-only
command's restart failure discloses no: a read has no side effect to repeat.
dispatch-disclosure.json: ios-runner.transport.written-then-lost keeps
unknown with a trigger that says the command is not resent;
ios-runner.transport.read-only-written-then-lost is new and discloses no.
Tests (runner-lifecycle-dispatch-disclosure, real connect loop over a
stubbed fetch and simctl curl exit 28, restart succeeds):
- written-then-lost: tap restarts the runner, is not sent on the restarted
runner, fails unknown with reason runner_reply_lost. Mutation: make
canResendAfterRestart return true -> fails (the tap is replayed and the
request succeeds).
- read-only-written-then-lost: snapshot is sent again on the restarted
runner and its failure discloses no. Mutations: drop the read-only term
from canResendAfterRestart -> fails (no resend); drop the read-only
branch of discloseRestartDispatch -> fails (unknown).
runner-command-retry: four mutating-restart fixtures now carry the proof a
real refused connect stamps (dispatched no); without it they would test the
removed replay.
* test(apple): share the unwritten connect refusal fixture to keep the retry test file within its size ratchet
* fix(apple-runner): report the restart transport fact for a read, not no
The restart path disclosed `no` for a read-only command whose first send
may have run, while the status-recovery path discloses `unknown` for the
same read (read_only_completed_without_retained_response, and the
notAccepted/unknown-state fallthrough). Both runner paths now report the
transport fact: a first attempt that may have written the command is
`unknown`, read or mutation. The read-only `no` has one owner, the daemon
router seam (request-dispatch-disclosure.ts), which keys it on the
registry's recordingEffect `observes-app` for every producer
(daemon.read-only-command). A second copy keyed on the runner's own
read-only trait reconstructed that rule from a different source of truth,
and the two runner paths disagreed. The recycle-budget refusal on the
restart path also stops deriving `no` from the read-only trait and uses
only whether the first attempt was provably unwritten.
The runner still resends a read on the restarted runner
(canResendAfterRestart); only the disclosure changes.
dispatch-disclosure.json:
- ios-runner.transport.read-only-written-then-lost: now `unknown`; the
trigger names the router rule that turns it into `no` for the caller.
- ios-runner.status.read-only-completed-without-retained-reply: new,
`unknown`, driven through the real connect loop and recovery module.
It pins the status-recovery read to the same value as the restart read.
Mutations:
- put back `if (isReadOnlyRunnerCommand(evidence.command)) return
discloseDispatch(error, 'no')` in discloseRestartDispatch ->
read-only-written-then-lost fails.
- set `dispatched: 'no'` on read_only_completed_without_retained_response
-> status.read-only-completed-without-retained-reply fails.
* test(apple-runner): drive a real runner death through the fake runner
runner-recovery-wiring: the notAccepted mutation test no longer scripts a
second `ok` for the mutation. Nothing consumed it, and it suggested that
a resend was expected. Its comment now says what the test models: status
answers notAccepted, the way a runner whose journal did not survive a
restart would. The path is status recovery, not the daemon's restart.
The fake runner server gains an `exit` reply: it hangs up and stops
listening, like a runner process that died mid-command. Before this, a
test could only hang up on one request while the same server stayed up.
Two new tests use a second server as the restarted runner:
- press/fill (runner tap/type): the runner dies on the command. The
mutation is sent once, is not sent to the restarted runner, fails with
dispatched unknown, and the dead session is invalidated. A readText on
the next request is served by the restarted runner. Mutation: declare
tap and type with READ_ONLY_TRAITS (the resend trait) -> both rows fail.
- get (runner readText): the runner dies mid-read. The connect loop gives
up with a possibly-written POST (simctl curl exit 28), the daemon
restarts the runner (runner_connect_failed_before_command_send), and the
read is resent once and succeeds. Mutation: reduce canResendAfterRestart
to `evidence.firstAttemptUnwritten` -> fails (no resend).
With the real transport, a mutating command never reaches
restartSessionAndRunCommand with a possibly-written first attempt.
Mutations are sent by sendRunnerCommandOnce, and only the connect loop
(waitForRunner, used for reads and the readiness preflight) stamps
runner_connect_refused. So the mutating arm of canResendAfterRestart is
covered only by the mocked lifecycle driver
(ios-runner.transport.written-then-lost). A dying runner sends a mutation
to status recovery, and a failed status probe rethrows the transport
error without a runner_reply_lost reason.
* fix(apple-runner): name the lost reply when the status probe fails
A runner that dies mid-mutation drops the reply, and the status probe that
follows fails. The error that reaches the caller was then a bare transport
error (`fetch failed`, dispatched unknown), with no typed reason. #3074
requires a reason that names the lost reply, so this path now builds an
error with reason runner_reply_lost (the name the restart arm already uses)
and recovery status_probe_failed. The error keeps the transport message
and details, and the transport error as cause. The session is still
invalidated, and the command is still not resent.
ADR 0011 gap list: the row ios-runner.transport.written-then-lost proves
the rule behind canResendAfterRestart's mutating arm, not a production
route. Over the real transport the arm cannot be reached, because
mutations go through sendRunnerCommandOnce and only the connect loop
raises the restart trigger.
Tests:
- runner-command-recovery: a failing status probe now asserts reason
runner_reply_lost, recovery status_probe_failed, dispatched unknown, and
the transport error as cause.
- runner-recovery-wiring: press/fill whose runner dies mid-command (real
fake-runner transport) now also asserts reason runner_reply_lost.
Mutation: drop `reason: RUNNER_REPLY_LOST_REASON` from
buildStatusProbeFailedError -> all three fail.
* test(apple-runner): hang up on every read attempt without counting the connect loop
The read-only lost-response driver scripted 20 hang-ups and a 400 ms
timeout. That count depended on the connect loop's retry cadence
(RUNNER_CONNECT_ATTEMPT_INTERVAL_MS): with a faster cadence the queue runs
out, and the fake answers the next attempt with `ok`. The fake runner
server now has a `hangUpAlways` reply that is never consumed, so the read
is refused until the loop gives up, however often it retries. The timeout
now only bounds how long the loop runs.
Mutation: set RUNNER_CONNECT_ATTEMPT_INTERVAL_MS to 10 or 50 (with a 1 ms
retry base delay) -> ios-runner.status.read-only-completed-without-retained-reply
still passes.
* test(apple-runner): pick the fake runner's next reply in one helper
The request handler of the fake runner server went over the fallow complexity threshold after hangUpAlways was added. The choice of the next scripted reply is now in takeScriptedResponse. Behavior is unchanged: the apple-runner suite passes (768 tests).
* refactor(apple-runner): gate the restart resend at the connect-failure call site
A mutation never reaches the connect loop, so the no-resend arm after a
restart had no production route. The call site now restarts only when the
first attempt provably wrote nothing or the command is read-only, and the
arm and its error builder are gone. The written-then-lost row is driven
through a fake runner that dies mid-command. A failed status probe keeps a
transport reason under transportReason.
* test(apple-runner): name runner_reply_lost in the lost-reply test titles
* test(apple-runner): the runner read-only set matches the registry's observes-app commands
* fix(apple-runner): decide the restart resend in the connect-failure classifier
shouldRestartRunnerBeforeCommandSend now takes the command and owns the
write-proof rule: a connect-loop failure restarts and resends only when no
attempt could have written the command, or the command is read-only. An
exit-28 POST that carries dispatched unknown no longer restarts a mutation,
whatever call site asks. The call-site guard in executeRunnerCommandAttempt
is gone.
* refactor(apple-runner): carry the first attempt's dispatch disclosure through the restart
restartSessionAndRunCommand takes firstAttemptDispatched: DispatchDisclosure
instead of a boolean that three ternaries decoded back into no or unknown.
resolveFirstAttemptDispatch computes it once beside
isRunnerCommandProvablyUnwritten, and the restart rule reads the same answer.
The readiness-preflight call site passes no.
* refactor(apple-runner): build every lost-reply failure in one place with one reason
Status recovery verdicts now carry either the runner's own answer or the
facts of a lost reply. applyRunnerTransportRecovery turns a lost reply into
one buildLostReplyError for a mutation, so runner_reply_lost is set on every
lost-reply verdict (completed without a retained reply, still in flight,
unknown or missing lifecycle, status probe failed, status unavailable) and
the transport's reason stays under transportReason. A read keeps its
transport error, which the read-only resend loop resends, so it never
carries the reason. The four hand-built copies are gone.
* fix(apple-runner): key a lost reply on its typed reason, never on transport text
* docs(adr): name the lost-reply dispatch-disclosure rows in ADR 0011
* test(apple-runner): pair keyboardReturn with the keyboard enter request that issues it
* test(apple-runner): pin the lost-reply branch in the connect-loop mutation test1 parent 7335a7f commit d37a9f4
14 files changed
Lines changed: 806 additions & 191 deletions
File tree
- contracts/fixtures
- docs/adr
- packages/platform-apple/src/runner
- __tests__
- src/__tests__
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
256 | 256 | | |
257 | 257 | | |
258 | 258 | | |
259 | | - | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
260 | 266 | | |
261 | 267 | | |
262 | 268 | | |
| |||
289 | 295 | | |
290 | 296 | | |
291 | 297 | | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
292 | 304 | | |
293 | 305 | | |
294 | 306 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
192 | 192 | | |
193 | 193 | | |
194 | 194 | | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
195 | 207 | | |
196 | 208 | | |
197 | 209 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
16 | | - | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
17 | 21 | | |
18 | 22 | | |
19 | 23 | | |
| |||
44 | 48 | | |
45 | 49 | | |
46 | 50 | | |
| 51 | + | |
47 | 52 | | |
48 | 53 | | |
49 | 54 | | |
| |||
53 | 58 | | |
54 | 59 | | |
55 | 60 | | |
56 | | - | |
57 | | - | |
58 | | - | |
59 | | - | |
60 | | - | |
61 | | - | |
62 | | - | |
63 | | - | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
64 | 67 | | |
| 68 | + | |
| 69 | + | |
65 | 70 | | |
66 | 71 | | |
67 | | - | |
68 | | - | |
69 | | - | |
70 | | - | |
71 | | - | |
72 | | - | |
73 | | - | |
| 72 | + | |
74 | 73 | | |
75 | 74 | | |
76 | 75 | | |
| |||
79 | 78 | | |
80 | 79 | | |
81 | 80 | | |
| 81 | + | |
82 | 82 | | |
83 | 83 | | |
84 | 84 | | |
85 | 85 | | |
86 | 86 | | |
87 | 87 | | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
88 | 116 | | |
89 | 117 | | |
90 | 118 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
| 9 | + | |
9 | 10 | | |
10 | 11 | | |
11 | 12 | | |
| |||
122 | 123 | | |
123 | 124 | | |
124 | 125 | | |
125 | | - | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
126 | 141 | | |
127 | 142 | | |
| 143 | + | |
128 | 144 | | |
129 | 145 | | |
130 | | - | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
131 | 157 | | |
132 | 158 | | |
133 | 159 | | |
134 | | - | |
| 160 | + | |
135 | 161 | | |
136 | 162 | | |
137 | 163 | | |
138 | 164 | | |
139 | 165 | | |
140 | | - | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
141 | 173 | | |
142 | 174 | | |
143 | 175 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
| 8 | + | |
8 | 9 | | |
9 | 10 | | |
10 | 11 | | |
| |||
49 | 50 | | |
50 | 51 | | |
51 | 52 | | |
| 53 | + | |
52 | 54 | | |
53 | 55 | | |
54 | 56 | | |
| |||
306 | 308 | | |
307 | 309 | | |
308 | 310 | | |
309 | | - | |
| 311 | + | |
310 | 312 | | |
311 | 313 | | |
312 | 314 | | |
| |||
329 | 331 | | |
330 | 332 | | |
331 | 333 | | |
332 | | - | |
| 334 | + | |
333 | 335 | | |
334 | 336 | | |
335 | 337 | | |
| |||
474 | 476 | | |
475 | 477 | | |
476 | 478 | | |
477 | | - | |
478 | 479 | | |
479 | | - | |
480 | | - | |
| 480 | + | |
| 481 | + | |
481 | 482 | | |
482 | 483 | | |
483 | 484 | | |
| |||
814 | 815 | | |
815 | 816 | | |
816 | 817 | | |
817 | | - | |
| 818 | + | |
818 | 819 | | |
819 | 820 | | |
820 | 821 | | |
| |||
970 | 971 | | |
971 | 972 | | |
972 | 973 | | |
973 | | - | |
974 | 974 | | |
975 | | - | |
976 | | - | |
| 975 | + | |
| 976 | + | |
977 | 977 | | |
978 | 978 | | |
979 | 979 | | |
| |||
1189 | 1189 | | |
1190 | 1190 | | |
1191 | 1191 | | |
1192 | | - | |
1193 | 1192 | | |
1194 | 1193 | | |
1195 | | - | |
| 1194 | + | |
1196 | 1195 | | |
1197 | 1196 | | |
1198 | 1197 | | |
| |||
0 commit comments