|
| 1 | +# Client lease heartbeats |
| 2 | + |
| 3 | +## Client-side work that precedes admission |
| 4 | + |
| 5 | +The [protection of admitted work](../../daemon/docs/remote-device-lease-contracts.md#admitted-request-work) covers nothing that happens before a request is admitted. |
| 6 | +An artifact install uploads the artifact from the caller before the request that will consume it is admitted. |
| 7 | +Thus, an upload slower than the inactivity window of the lease expired the lease that paid for the target device of the bytes (#2946). |
| 8 | +The caller therefore sends lease heartbeats over the ordinary transport for as long as that phase runs. |
| 9 | +Each heartbeat names the lease scope exactly as the command named it, and carries no payload of its own. |
| 10 | +An install names no window, so its heartbeats renew for the window that the lease already carries. |
| 11 | +A caller that did name a window continues to renew on that window. |
| 12 | + |
| 13 | +The first heartbeat is sent when the phase starts, not one cadence later. |
| 14 | +The reason is that a heartbeat proves that the lease on which the upload spends time is still alive. |
| 15 | +Otherwise, a lease shorter than any fixed cadence would lapse before anything renewed it. |
| 16 | +A lease that is already gone is caught early, but not before the phase begins. |
| 17 | +Hashing, the preflight, and the start of the stream can all run while the first heartbeat is still outstanding. |
| 18 | +Thus, the first heartbeat makes sure that the loss is learned during the upload instead of after it. |
| 19 | +Each heartbeat answers with the window that it just renewed. |
| 20 | +The loop arms the next heartbeat when the current heartbeat starts, not when it settles. |
| 21 | +Thus, a heartbeat that never answers is abandoned on schedule and does not stop the schedule. |
| 22 | +A phase that settles does not wait for a heartbeat still in flight. |
| 23 | + |
| 24 | +Cadence and budget are deliberately two numbers. |
| 25 | +A heartbeat is armed every third of the window, but it is allowed the whole window to answer. |
| 26 | +If the budget were equal to the cadence, a heartbeat that needs more than a third of a window for its round trip could never answer. |
| 27 | +Examples are a tunneled proxy and an uplink that the upload itself saturates. |
| 28 | +Every heartbeat would then time out again on exactly the slow links that the heartbeat exists to protect. |
| 29 | +The window, not the cadence, is the interval in which liveness still matters. |
| 30 | +Thus, the window is also the longest time that an answer to a heartbeat is worth a wait. |
| 31 | + |
| 32 | +That ceiling is the shorter of the window and the heartbeat's own request policy, currently ninety seconds. |
| 33 | +Thus, a ten-minute lease gives a heartbeat ninety seconds instead of the full window. |
| 34 | +This limit costs nothing that the heartbeat exists to provide. |
| 35 | +The daemon renews the lease when it handles the request, before it answers. |
| 36 | +Thus, a round trip abandoned at the cap has still extended the lease if the request ever arrived. |
| 37 | +If the request never arrived, the transport has stopped carrying traffic. |
| 38 | +A socket held open for the rest of a window measured in minutes does not help that transport. |
| 39 | +The cap refuses exactly that idle hold. |
| 40 | +The cap binds only on windows longer than the policy. |
| 41 | +On those windows, the heartbeat already has the longest answer deadline that any lease gets. |
| 42 | +Shorter windows are bounded by their own window, as above. |
| 43 | + |
| 44 | +A heartbeat is a fresh request each time, never the protected request rewritten. |
| 45 | +The reason is that a timed-out heartbeat is canceled under its request identity, and heartbeats must not inherit each other's cancellation. |
| 46 | +In these cases, a heartbeat ends the phase with that error and cancels the upload: |
| 47 | + |
| 48 | +- The heartbeat returns `UNAUTHORIZED` with `LEASE_NOT_FOUND`, `LEASE_EXPIRED`, or `LEASE_REVOKED`. |
| 49 | +- The heartbeat returns `COMMAND_FAILED` with `provider-instance-ended`. |
| 50 | +- The heartbeat returns an `AppError` with `LEASE_SCOPE_REQUIRED` or `LEASE_SCOPE_MISMATCH`. |
| 51 | +- The heartbeat returns an `AppError` with code `INVALID_ARGS`. |
| 52 | + |
| 53 | +When the provider reports `provider-instance-ended`, the daemon ends the lease and its sessions. |
| 54 | +A failed session teardown does not replace that typed reason. |
| 55 | + |
| 56 | +Thus, the upload does not finish bytes against a device that nobody owns or a lease that will stop renewing. |
| 57 | +A heartbeat that fails for any other reason is reported, and the phase continues. |
| 58 | +The reason is that a later heartbeat covers one lost request. |
| 59 | +This includes a request abandoned at its budget instead of answered. |
| 60 | +Thus, that promise holds for a stalled round trip, not only for a round trip that fails fast. |
0 commit comments