[Ubuntu 26.04][Agent&Skills] hermes chat stalls after host network change (stale sandbox DNS) #8219

Description

After the host machine's network changes (e.g. laptop switched from home Wi-Fi to office network) without restarting or reconnecting the sandbox container, the Hermes CLI's built-in "3 retries with backoff" wraps each upstream call in a fresh DNS lookup. When the container's /etc/resolv.conf still points at the pre-switch resolvers (Docker snapshots resolv.conf at container creation and does not refresh it when the host's DNS changes), every retry stalls on DNS timeouts. A single hermes chat --query grows from a few seconds to ~67 s, and a typical multi-turn interactive session (~15-20 tool-using turns) compounds to ~20 min end-to-end. The original user report showed the same behaviour independent of the configured provider — both a cloud provider (api.deepseek.com) and a locally-deployed Ollama model exhibited the slowdown, confirming the delay is in the shared retry+DNS layer rather than any single provider.

Platform scope: Reproduced on Ubuntu 26.04 (x86_64) only; other platforms not tested.
Regression: Unknown — earlier versions not tested.

Environment

Device:        Ubuntu 26.04 LTS server (x86_64)
OS:            Ubuntu 26.04 LTS (kernel 7.0.0-27-generic)
Architecture:  x86_64
Node.js:       v22.23.1
npm:           10.9.8
Docker:        29.6.1
OpenShell CLI: 0.0.72
NemoClaw:      v0.0.101
OpenClaw:      Hermes agent v0.17.0 (2026.6.19)

Steps to Reproduce

1. On a healthy host with working DNS, onboard a Hermes sandbox (following https://docs.nvidia.com/nemoclaw/latest/get-started/quickstart-hermes.html) and confirm hermes chat responds within a few seconds.

2. Simulate a network change without restarting the sandbox container — the simplest deterministic proxy for the real "home to office" transition is to point the container's /etc/resolv.conf at an unreachable nameserver:

   docker exec -u root {container id} sh -c \
     'cp /etc/resolv.conf /tmp/resolv.bak && \
      printf "nameserver 203.0.113.10\noptions timeout:2 attempts:2 ndots:0\n" > /etc/resolv.conf'

3. Measure a single hermes chat inside the sandbox:

   time docker exec -u sandbox {container id} sh -c \
     'cd /sandbox && HOME=/sandbox hermes chat --query "reply just OK"'

4. Restore afterwards:

   docker exec -u root {container id} cp /tmp/resolv.bak /etc/resolv.conf

Expected Result

A single hermes chat turn should complete in a few seconds when the previous session worked, or fail fast (single clear "DNS timeout" / "cannot resolve host" message within a small budget) when DNS is stale. It should not silently retry three times, doing full DNS resolution on each attempt, before giving up.

Actual Result

Under stale DNS the same chat query takes ~67 s to fail — Hermes prints three retry rows whose Elapsed timing grows because each retry re-resolves the upstream host:

• Attempt 1/3: APIConnectionError — Elapsed 31.23 s (~2 DNS lookups × ~8 s each + connect)
• Attempt 2/3: APIConnectionError — Elapsed 38.26 s
• Attempt 3/3: APIConnectionError — Elapsed 46.69 s
• Total wall time for one query: 1 min 7 s

Under a healthy resolver on the same container the same failing call takes ~29 s (Elapsed 13 / 16 / 20 s) — so stale DNS more than doubles per-query latency. Multiplied across a typical multi-turn Hermes session (~15-20 tool-using turns), the end-to-end wait matches the user-reported ~20 minutes.

The measured DNS timeout inside the container under the stale configuration is ~8 s per hostname lookup (time getent hosts api.deepseek.com and time getent hosts inference.local both return in ~8.07 s). This matches Hermes' per-attempt overhead pattern.

Logs

Stale-DNS reproduction (single hermes chat, 3 retries):

$ time getent hosts api.deepseek.com
real    0m8.063s

$ time getent hosts inference.local
real    0m8.075s

$ time timeout 120 hermes chat --query "just say OK"
   Provider: custom  Model: nvidia/nemotron-3-super-120b-a12b
   Endpoint: https://inference.local/v1
   Error: Connection error.
   Elapsed: 31.23s  Context: 2 msgs, ~4,246 tokens
Retrying in 2.8s (attempt 1/3)...

   Provider: custom  Model: nvidia/nemotron-3-super-120b-a12b
   Endpoint: https://inference.local/v1
   Error: Connection error.
   Elapsed: 38.26s
Retrying in 4.3s (attempt 2/3)...

   Provider: custom  Model: nvidia/nemotron-3-super-120b-a12b
   Endpoint: https://inference.local/v1
   Error: Connection error.
   Elapsed: 46.69s
API failed after 3 retries — Connection error.

Session: 20260804_105509_f5d759
Duration: 1m 4s
real    1m 7.121s

Baseline (embedded Docker DNS, same broken upstream) for comparison — same
3-retry pattern but each attempt Elapsed 13.62 / 16.04 / 20.47 s, total 29 s.