Repository navigation
SystemdUser: Persistent=true runs missed jobs at boot before the network is up, so network jobs fail on every boot #2785
Description
What happens
On Linux, LIFEOS/TOOLS/lib/SystemdUser.ts renders each scheduled job as a systemd --user oneshot service plus a timer with Persistent=true. When the machine was off at the scheduled time, Persistent=true runs the missed job as soon as the user manager starts at boot. That is before NetworkManager has connected, so every job that calls the network fails on the first run after each boot.
Observed
Over 12 days, com.lifeos.commitmentsweep and com.lifeos.modelsync failed on each of the three boot catch-up runs and on no other run:
| Event | Time |
|---|---|
| user manager starts both jobs | 08:20:21 |
both exit 1 (error connecting to api.github.com) |
08:20:22 |
NetworkManager: state is now CONNECTED_GLOBAL |
08:20:28 |
User units cannot order themselves after the system network-online.target, because the user manager does not see system targets.
Fix we run locally
renderService adds this line to every calendar (OnCalendar=) unit, only when nm-online is found on the machine at install time:
ExecStartPre=-/usr/bin/nm-online -q -t 60
When the network is already up, it returns at once. At boot, it waits the few seconds until NetworkManager connects. The leading - lets a 60 s timeout fall through to the job, which then fails visibly as before. Interval units and daemons are left as they were: offline, an interval job every few minutes would wait 60 s on every run, and none of ours failed at boot. Machines without NetworkManager (for example macOS, or systemd-networkd) render the unit unchanged.
Activity
gjaworski86 commentedon Nov 13, 2026
Note
🤖 Klaudia (AI assistant) responding on behalf of Grzegorz
The same thing happens on macOS, through a different path. There the catch-up comes from the Workflows dispatcher: a schedule trigger with
missed_run: onceis submitted on the first dispatcher pass after boot.One boot here, machine off overnight:
errata-dailyandassay-engine-fingerprintnetwork(344 calls)Other catch-up runs in the same pass that need no network succeeded. Both failed jobs passed when run by hand half an hour later.
So a fix in
SystemdUser.tsalone leaves macOS as it is. A network check in the dispatcher before it submits a catch-up run would cover both platforms.gjaworski86 commentedon Nov 13, 2026
Note
🤖 Klaudia (AI assistant) responding on behalf of Grzegorz
A third path, same boot as my comment above: launchd
RunAtLoad. The codexupdate template shipsRunAtLoad=true(added for #1513), so the agent runs at login:com.lifeos.codexupdaterunserror: DNSResolveFailed downloading package manifest @openai/codex→ exit 1A manual run of the same command later exited 0.
launchctl listkeeps status 1 until the next 04:00 run, so a monitor reading it stays red for a day.CodexUpdate.tsadds to it: when the registry fetch fails (:53) it falls through tobun install -g, which fails the same way after one attempt (:71-89).So a catch-up run at boot, before the network is up, now fails on three paths: systemd
Persistent=true, the Workflows dispatchermissed_run: once, and launchdRunAtLoad. One shared wait-for-network step at the start of every network-bound scheduled job (return at once when online, wait up to ~60 s at boot, then fall through and fail visibly as today) would cover all three, the waynm-onlinedoes in the fix above.