Skip to content

Commit a0bf68e

Browse files
nik13claude
andcommitted
docs(devin-desktop): troubleshooting, uninstall and attribute wording from recheck (TH-8294)
Page: - Troubleshooting step 1: "no lines" also means the emitter found no keys (env file missing or misnamed, keys only in the shell) or FI_HOOK_DISABLED=1; the turn start alone never logs. - Troubleshooting step 2: the manual run sends one test trace (session fi-hook-test, one read_code span); prefix FI_HOOK_DISABLED=1 to check the emitter without sending. FI_HOOK_DUMP_FILE is not offered because it does not stop the send. - Common fields are on every tool and turn span; the Cascade session span has no turn id and carries the model as llm.model_name. - The 24 h state sweep runs at the next prompt; Uninstall now deletes the state directory and, if unused, ~/.config/fi-hook-to-otlp/env. - install.sh prints the same uninstall steps. Tests: the page's manual-run command (disabled: nothing sent, logged or written; enabled: exactly that test trace, one http=200 line), no keys or FI_HOOK_DISABLED=1 log nothing, the page's Uninstall block removes the emitter, the TMPDIR state dir and the env file, and common fields are on tool and turn spans but not on the session root. Co-Authored-By: Claude Opus 5.5 <[email protected]>
1 parent 931dac6 commit a0bf68e

4 files changed

Lines changed: 163 additions & 15 deletions

File tree

‎examples/coding-agents/devin-desktop/install.sh‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,4 +32,6 @@ echo
3232
echo "Credentials: put FI_API_KEY, FI_SECRET_KEY and FI_PROJECT_NAME in"
3333
echo "~/.config/fi-hook-to-otlp/env (chmod 600). Devin Desktop started from the"
3434
echo "Dock may not see variables exported in your shell."
35-
echo "To uninstall: remove the entries from $HOOK_DIR/hooks.json first, then delete $DEST."
35+
echo "To uninstall: remove the entries from $HOOK_DIR/hooks.json first, then delete $DEST,"
36+
echo "the state files in \${TMPDIR:-/tmp}/fi-hook-to-otlp-$(id -u)/devin_desktop and, if no other"
37+
echo "app's hook uses it, ~/.config/fi-hook-to-otlp/env."

‎examples/coding-agents/hook-to-otlp/tests/test_devin_desktop_adapter.py‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,30 @@ def test_tool_metadata(self):
6767
self.assertEqual(write["devin_desktop.model_label"], "Claude Sonnet 4")
6868
self.assertNotIn("devin_desktop.action", write) # duplicates coding_agent.hook.event
6969

70+
def test_common_fields_are_on_tool_and_turn_spans_not_the_root(self):
71+
# The page: every tool and turn span carries the hook name, the turn id and the model
72+
# label; the Cascade session span has no turn id and carries the model as llm.model_name.
73+
spans = self.spans()
74+
root_id = emit.span_id("devin_desktop", SID, SESSION, "")
75+
known_model_turn = "fixture-exec-0001"
76+
for span in spans:
77+
a = attrs(span)
78+
self.assertIn("coding_agent.hook.event", a, span["name"])
79+
if span["spanId"] == root_id:
80+
self.assertNotIn("coding_agent.turn.id", a)
81+
self.assertNotIn("devin_desktop.model_label", a)
82+
self.assertEqual(a["llm.model_name"], "Claude Sonnet 4")
83+
continue
84+
self.assertIn("coding_agent.turn.id", a, span["name"])
85+
if a["coding_agent.turn.id"] == known_model_turn:
86+
self.assertEqual(a["devin_desktop.model_label"], "Claude Sonnet 4", span["name"])
87+
if span["name"] == "Cascade turn":
88+
self.assertEqual(a["llm.model_name"], "Claude Sonnet 4")
89+
tools = [s for s in spans if s["spanId"] != root_id and s["name"] != "Cascade turn"]
90+
self.assertTrue(tools)
91+
for span in tools:
92+
self.assertNotIn("llm.model_name", attrs(span), span["name"])
93+
7094
def test_tool_status(self):
7195
spans = self.spans()
7296
# Read, write and MCP hooks fire after a successful action; a command's outcome is unknown.

‎examples/coding-agents/hook-to-otlp/tests/test_devin_desktop_hooks_config.py‎

Lines changed: 116 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -17,13 +17,23 @@
1717
import shutil
1818
import subprocess
1919
import tempfile
20+
import textwrap
2021
import unittest
2122
from pathlib import Path
2223

23-
from tests.helpers import ROOT, Receiver
24+
from tests.helpers import ROOT, Receiver, all_spans
2425

2526
APP_DIR = ROOT.parent / "devin-desktop"
2627
HOOKS = json.loads((APP_DIR / "hooks.json").read_text(encoding="utf-8"))["hooks"]
28+
PAGE = (ROOT.parents[2] / "src/pages/docs/integrations/coding-agents/devin-desktop.mdx").read_text(encoding="utf-8")
29+
BASH = shutil.which("bash") or "/bin/bash"
30+
31+
32+
def page_bash_block(after: str) -> str:
33+
"""The first ```bash block on the page after the heading or text ``after``, dedented."""
34+
section = PAGE.split(after, 1)[1]
35+
block = section.split("```bash\n", 1)[1].split("```", 1)[0]
36+
return textwrap.dedent(block)
2737
EXPECTED = {"pre_user_prompt", "post_read_code", "post_write_code", "post_run_command",
2838
"post_mcp_tool_use", "post_cascade_response"}
2939
# Cascade uses bash; /bin/sh is the POSIX baseline.
@@ -71,8 +81,7 @@ def test_registered_hooks_are_guarded(self):
7181
self.assertTrue(entry["command"].endswith(f" devin_desktop {event} 2>/dev/null || true)"), event)
7282

7383
def test_page_shows_the_shipped_hooks_json(self):
74-
page = (ROOT.parents[2] / "src/pages/docs/integrations/coding-agents/devin-desktop.mdx").read_text(encoding="utf-8")
75-
block = page.split("```json\n", 1)[1].split("\n```", 1)[0]
84+
block = PAGE.split("```json\n", 1)[1].split("\n```", 1)[0]
7685
self.assertEqual(json.loads(block)["hooks"], HOOKS)
7786

7887
def test_missing_emitter_never_blocks(self):
@@ -95,11 +104,111 @@ def test_missing_python_never_blocks(self):
95104
finally:
96105
shutil.rmtree(empty_bin, ignore_errors=True)
97106

107+
def install(self) -> subprocess.CompletedProcess:
108+
result = subprocess.run([BASH, str(APP_DIR / "install.sh")], capture_output=True, text=True,
109+
env={"PATH": os.environ.get("PATH", ""), "HOME": self.home}, timeout=30)
110+
self.assertEqual(result.returncode, 0, result.stderr)
111+
return result
112+
113+
def write_env_file(self, receiver: Receiver) -> Path:
114+
"""The page's env file, pointed at a local receiver instead of Future AGI."""
115+
env_file = Path(self.home, ".config/fi-hook-to-otlp/env")
116+
env_file.parent.mkdir(parents=True)
117+
env_file.write_text(f"FI_API_KEY=k\nFI_SECRET_KEY=s\nFI_HOOK_SYNC=1\n"
118+
f"FI_OTLP_ENDPOINT={receiver.url}/v1/traces\n"
119+
f"FI_HOOK_LOG_FILE={self.home}/fi-hook.log\n", encoding="utf-8")
120+
return env_file
121+
122+
def tmpdir(self) -> str:
123+
"""A TMPDIR with a trailing slash, as macOS sets it."""
124+
path = os.path.join(self.home, "tmp")
125+
os.mkdir(path)
126+
return path + "/"
127+
128+
def log_lines(self) -> list:
129+
log = Path(self.home, "fi-hook.log")
130+
return log.read_text(encoding="utf-8").splitlines() if log.exists() else []
131+
132+
def test_page_manual_hook_run_sends_one_test_trace_unless_disabled(self):
133+
self.install()
134+
receiver = Receiver()
135+
try:
136+
self.write_env_file(receiver)
137+
tmp = self.tmpdir()
138+
command = page_bash_block("Run one hook by hand")
139+
self.assertIn("put `FI_HOOK_DISABLED=1` in front of `python3`", PAGE)
140+
disabled = command.replace("| python3 ", "| FI_HOOK_DISABLED=1 python3 ", 1)
141+
self.assertNotEqual(disabled, command)
142+
result = run(disabled, BASH, "stdin", self.home, "", TMPDIR=tmp)
143+
self.assertEqual((result.returncode, result.stdout, result.stderr), (0, "", ""))
144+
self.assertEqual((receiver.requests, self.log_lines()), ([], []), "disabled: nothing sent or logged")
145+
self.assertEqual(list(Path(tmp).rglob("*.json")), [], "disabled: no state written")
146+
147+
result = run(command, BASH, "stdin", self.home, "", TMPDIR=tmp)
148+
self.assertEqual((result.returncode, result.stdout, result.stderr), (0, "", ""))
149+
receiver.wait_for(1)
150+
self.assertEqual(len(receiver.requests), 1)
151+
spans = all_spans(receiver.requests[0][2])
152+
self.assertEqual(sorted(s["name"] for s in spans), ["Cascade session", "read_code"])
153+
self.assertIn("one test trace to your project (a **Cascade session** with session id `fi-hook-test`"
154+
" and one **read_code** span)", PAGE)
155+
for span in spans:
156+
sid = [a["value"]["stringValue"] for a in span["attributes"] if a["key"] == "session.id"]
157+
self.assertEqual(sid, ["fi-hook-test"], span["name"])
158+
lines = self.log_lines()
159+
self.assertEqual(len(lines), 1)
160+
self.assertIn("http=200", lines[0])
161+
finally:
162+
receiver.close()
163+
164+
def test_no_keys_means_no_log_lines(self):
165+
# Troubleshooting step 1: without keys (or with FI_HOOK_DISABLED=1) the emitter logs nothing.
166+
self.install()
167+
receiver = Receiver()
168+
try:
169+
env_file = self.write_env_file(receiver)
170+
keyed = env_file.read_text(encoding="utf-8")
171+
command = HOOKS["post_read_code"][0]["command"]
172+
for label, text in (("no keys", keyed.replace("FI_API_KEY=k\nFI_SECRET_KEY=s\n", "")),
173+
("disabled", keyed + "FI_HOOK_DISABLED=1\n")):
174+
env_file.write_text(text, encoding="utf-8")
175+
result = run(command, BASH, "stdin", self.home, payload("post_read_code"),
176+
FI_HOOK_STATE_DIR=os.path.join(self.home, "state"))
177+
self.assertEqual((result.returncode, result.stdout), (0, ""), label)
178+
self.assertEqual((receiver.requests, self.log_lines()), ([], []), label)
179+
finally:
180+
receiver.close()
181+
182+
def test_page_uninstall_removes_emitter_state_and_keys(self):
183+
install = self.install()
184+
self.assertIn("~/.config/fi-hook-to-otlp/env", install.stdout)
185+
self.assertIn(f"fi-hook-to-otlp-{os.getuid()}/devin_desktop", install.stdout)
186+
receiver = Receiver()
187+
try:
188+
env_file = self.write_env_file(receiver)
189+
tmp = self.tmpdir()
190+
for event in ("pre_user_prompt", "post_read_code"): # default state dir under TMPDIR
191+
result = run(HOOKS[event][0]["command"], BASH, "stdin", self.home, payload(event), TMPDIR=tmp)
192+
self.assertEqual(result.returncode, 0, event)
193+
finally:
194+
receiver.close()
195+
state_dir = Path(tmp, f"fi-hook-to-otlp-{os.getuid()}", "devin_desktop")
196+
self.assertTrue(list(state_dir.glob("*.json")), "a hook left a state file")
197+
emitter_dir = Path(self.home, ".codeium/windsurf/fi-hook-to-otlp")
198+
self.assertTrue(emitter_dir.is_dir())
199+
200+
result = run(page_bash_block("## Uninstall"), BASH, "stdin", self.home, "", TMPDIR=tmp)
201+
self.assertEqual((result.returncode, result.stderr), (0, ""))
202+
self.assertFalse(emitter_dir.exists())
203+
self.assertFalse(state_dir.exists())
204+
self.assertFalse(env_file.exists())
205+
self.assertTrue(Path(self.home, ".codeium/windsurf/hooks.json").exists(), "hooks.json is edited by hand")
206+
# A second run (nothing left to delete) still succeeds.
207+
result = run(page_bash_block("## Uninstall"), BASH, "stdin", self.home, "", TMPDIR=tmp)
208+
self.assertEqual(result.returncode, 0, result.stderr)
209+
98210
def test_installed_commands_read_the_payload_and_print_nothing(self):
99-
install = subprocess.run([shutil.which("bash") or "/bin/bash", str(APP_DIR / "install.sh")],
100-
capture_output=True, text=True,
101-
env={"PATH": os.environ.get("PATH", ""), "HOME": self.home}, timeout=30)
102-
self.assertEqual(install.returncode, 0, install.stderr)
211+
self.install()
103212
installed = json.loads(Path(self.home, ".codeium/windsurf/hooks.json").read_text())["hooks"]
104213
self.assertEqual(installed, HOOKS)
105214
for shell in SHELLS:

‎src/pages/docs/integrations/coding-agents/devin-desktop.mdx‎

Lines changed: 20 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -92,15 +92,15 @@ Open **Observe** in Future AGI and select the project (`cascade-sessions` above)
9292
| `post_mcp_tool_use` | tool span named after `mcp_tool_name`, status OK | MCP server and tool name |
9393
| `post_cascade_response` | **Cascade turn** end | |
9494

95-
Every span also carries the hook name (`coding_agent.hook.event`), the turn's `execution_id` (`coding_agent.turn.id`), and the model label. Cascade fires the read, write, and MCP post-hooks after a successful action, so those spans are OK. `post_run_command` carries no exit code, so a command span's status is left unset rather than shown as successful.
95+
Every tool and turn span also carries the hook name (`coding_agent.hook.event`), the turn's `execution_id` (`coding_agent.turn.id`), and the model label (`devin_desktop.model_label`). The **Cascade session** span has no turn id; it carries the model as `llm.model_name`. Cascade fires the read, write, and MCP post-hooks after a successful action, so those spans are OK. `post_run_command` carries no exit code, so a command span's status is left unset rather than shown as successful.
9696

97-
The model is Cascade's `model_name`, the label shown in the model selector (for example `Claude Sonnet 4`). It is recorded as `llm.model_name` and `devin_desktop.model_label`, but it is a display name that Devin Desktop may change, not a stable model id. When Cascade reports `Unknown`, no model is recorded.
97+
The model is Cascade's `model_name`, the label shown in the model selector (for example `Claude Sonnet 4`). It is recorded as `llm.model_name` on the session and turn spans and as `devin_desktop.model_label` on the turn and tool spans. It is a display name that Devin Desktop may change, not a stable model id. When Cascade reports `Unknown`, no model is recorded.
9898

9999
## What is not available
100100

101101
- **Token counts, cost, and model request spans.** Cascade hooks do not report them.
102102
- **Tool duration, tool call ids, and command exit codes.** Post-hooks carry none of them, so tool spans mark when the action finished and a failed command is not marked as an error.
103-
- **Session start and end.** Cascade has no session hook. The session span starts at the first prompt and is re-sent, with a later end time, whenever a turn or tool span is sent. The emitter's per-conversation state file is therefore not deleted at the end of a conversation; it is swept after 24 hours (`FI_HOOK_STATE_TTL_HOURS`).
103+
- **Session start and end.** Cascade has no session hook. The session span starts at the first prompt and is re-sent, with a later end time, whenever a turn or tool span is sent. The emitter's per-conversation state file is therefore not deleted at the end of a conversation. Instead, each new prompt (in any conversation) deletes state files unused for 24 hours (`FI_HOOK_STATE_TTL_HOURS`). The sweep runs only when a hook runs, so see [Uninstall](#uninstall) for the files left behind when you remove the hooks.
104104
- **Turn status.** There is no hook for a cancelled or failed turn. If Cascade never sends `post_cascade_response` for a turn, the emitter closes that turn when your next prompt starts and marks it `coding_agent.incomplete=true`; if the response hook arrives later after all, the turn is re-sent complete.
105105
- **Devin Local and hosted Devin.** See [About](#about).
106106

@@ -120,20 +120,33 @@ Devin Desktop 3.8.20 was downloaded and its bundled Cascade language server was
120120
## Troubleshooting
121121

122122
**No trace appears.**
123-
1. Set `FI_HOOK_LOG_FILE=/tmp/fi-hook.log` in `~/.config/fi-hook-to-otlp/env` and send another prompt. No lines: Cascade did not run the command (check that `python3` resolves and the emitter path exists, and that the workspace is not in Restricted Mode). `error=` lines: the collector is unreachable. `http=401`/`403`: wrong keys.
123+
1. Set `FI_HOOK_LOG_FILE=/tmp/fi-hook.log` in `~/.config/fi-hook-to-otlp/env`, send another prompt, and let Cascade finish. Each hook that sends spans writes one line; the turn start alone writes none.
124+
- No lines: either Cascade did not run the command (check that `python3` resolves, that the emitter path exists, and that the workspace is not in Restricted Mode), or the emitter is not sending. It sends and logs nothing when it finds no keys or `FI_HOOK_DISABLED=1` is set. Check that `FI_API_KEY` and `FI_SECRET_KEY` are in `~/.config/fi-hook-to-otlp/env` at exactly that path, because the app may not see keys exported only in your shell.
125+
- `error=` lines: the collector is unreachable.
126+
- `http=401`/`403`: wrong keys.
124127
2. Run one hook by hand. The guard hides errors, so drop it while testing:
125128
```bash
126-
echo '{"agent_action_name":"post_read_code","trajectory_id":"t1","execution_id":"e1","tool_info":{}}' \
129+
echo '{"agent_action_name":"post_read_code","trajectory_id":"fi-hook-test","execution_id":"e1","tool_info":{}}' \
127130
| python3 -B ~/.codeium/windsurf/fi-hook-to-otlp/emit.py devin_desktop post_read_code
128131
```
129-
It should exit 0 without output. A `can't open file` error means the emitter is not installed at that path.
132+
It should exit 0 without output. A `can't open file` error means the emitter is not installed at that path. With keys in the env file, this sends one test trace to your project (a **Cascade session** with session id `fi-hook-test` and one **read_code** span) and writes an `http=` line to the log from step 1. To check only that the emitter runs, without sending anything, put `FI_HOOK_DISABLED=1` in front of `python3`.
130133
3. If your organization also deploys hooks from the Devin Desktop dashboard or a system-level `hooks.json`, they run before yours (dashboard, system, user, then workspace).
131134

132135
**A turn is marked incomplete.** Cascade did not send `post_cascade_response` for that turn before the next prompt (for example, the turn was cancelled). If the last turn of a conversation has no turn span at all, check that `post_cascade_response` is in `hooks.json`: Cascade has no session-end hook to close it.
133136

134137
## Uninstall
135138

136-
Remove the Future AGI entries from `~/.codeium/windsurf/hooks.json` first, then delete `~/.codeium/windsurf/fi-hook-to-otlp/`. To pause tracing without editing the hook file, set `FI_HOOK_DISABLED=1` in the env file.
139+
Remove the Future AGI entries from `~/.codeium/windsurf/hooks.json` first. Then delete the emitter and the files it leaves behind. Once the hooks stop running, nothing sweeps the state files:
140+
141+
```bash
142+
rm -rf ~/.codeium/windsurf/fi-hook-to-otlp # the emitter
143+
rm -rf "${TMPDIR:-/tmp}/fi-hook-to-otlp-$(id -u)/devin_desktop" # per-conversation state files
144+
rm -f ~/.config/fi-hook-to-otlp/env # your keys; keep it if another app's hook still uses it
145+
```
146+
147+
The state files hold turn ids, the model label, and timing. While content capture is on, they also hold the open turn's prompt and the content of turns closed as incomplete. If you set `FI_HOOK_STATE_DIR`, they are in `$FI_HOOK_STATE_DIR/devin_desktop` instead.
148+
149+
To pause tracing without editing the hook file, set `FI_HOOK_DISABLED=1` in the env file.
137150

138151
## Next Steps
139152

0 commit comments

Comments
 (0)