Skip to content

Commit 7205175

Browse files
Tom039224claude
andcommitted
test: 実 OpenCode のテストを上流バグ回避のウォームアップシム経由にし、想定外の失敗で即失敗させる
OpenCode 2.0.18 は opencode acp の最初の session/new のモデル一覧が不完全で model not found になる (anomalyco/opencode#50236)。Yhtye 本体は回避せず、 テスト専用のシム opencode-warmup-shim (yhtye-fake-agent の bin) が捨ての session/new を先に流す。実 Agent のオーケストレーションテストは SessionFailed / AgentCrashed で全体のタイムアウトを待たず即失敗する。 Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
1 parent c6d4e7a commit 7205175

10 files changed

Lines changed: 348 additions & 38 deletions

File tree

‎PLAN.md‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -959,7 +959,7 @@ Stage 7a で以下をユーザーが決定した (経緯は各 Stage の結果
959959
(Haiku のオーケストレータが行に無い `effort=high` を指定 → 行 + 用途メモ付きで拒否 → 完全一致の行で作り直し、Haiku で起動して main にマージ)。
960960
- 実 OpenCode (`opencode/muse-spark-1.3-contributor-free`): `acp_opencode_real::real_opencode_efforts_are_read_per_model_and_applied` (variant =
961961
minimal/low/medium/high/xhigh を約 1.2 秒で読み、`minimal` を設定 → `effort=minimal` を報告)。
962-
- **実 OpenCode を含む混成実行は今は再現できない (環境の問題)**: 開発中に OpenCode が 2.0.12 → 2.0.18 に更新され、この環境では **git リポジトリの cwd で
962+
- **[解決済み: 下の 2026-09-29 の追記]** **実 OpenCode を含む混成実行は今は再現できない (環境の問題)**: 開発中に OpenCode が 2.0.12 → 2.0.18 に更新され、この環境では **git リポジトリの cwd で
963963
`opencode acp` に無料モデルを `set_config_option` すると `model not found` になる** (git 以外のディレクトリでは通る。最小の再現: 一時 git リポジトリ +
964964
`HarnessPreset::opencode` の設定で起動。`opencode models` にも muse-spark が出ない)。7c-2 の `orchestration_mixed_real` 既存 2 件も同じ理由で今は通らない
965965
(7d の変更とは無関係)。追加した `orchestration_mixed_real::real_effort_rows_are_matched_exactly_and_applied` は、OpenCode が使える状態だった開発序盤の
@@ -969,5 +969,13 @@ Stage 7a で以下をユーザーが決定した (経緯は各 Stage の結果
969969
(`4-gear-at-bottom-of-rail.jpg`)。設定モーダルで実装に行を追加 → sonnet + High + 用途メモ (`1-settings-modal-effort-rows.jpg`、DB の `agent_settings` に
970970
effort・note が保存されたことを確認)。OpenCode の行で検索付きピッカー (`2-opencode-model-picker.jpg`)、無料モデルを選ぶと effort 一覧
971971
(minimal〜xhigh) を `list_model_efforts` で取得 (`3-opencode-lazy-efforts.jpg`)。Esc で閉じる。ブリッジ・Vite は停止済み。
972+
- 2026-09-29 追記 (OpenCode の環境問題の原因と対処): 原因は OpenCode 2.0.18 の上流バグ anomalyco/opencode#50236 (修正 PR #50619 未マージ) —
973+
`opencode acp` の**最初の `session/new`** のモデル一覧にプロバイダ読み込み前のスナップショットが使われ `opencode/*` が無く、`model not found` になる
974+
(git リポジトリかどうかは無関係だった)。**Yhtye 本体では回避しない (ユーザー決定)**。実機テストだけ、`crates/yhtye-fake-agent` の
975+
テスト用シム `opencode-warmup-shim` (捨ての `session/new` を 1 回先に流す) 経由で `opencode acp` を起動する。詳細は `acp-harnesses.md` §7.7。
976+
実 OpenCode を含むテストのヘルパーは `tests/common/opencode.rs::{opencode_harness, opencode_preset}`。実 Agent のオーケストレーションテストは、想定外の
977+
`SessionFailed` / `AgentCrashed` の help で全体のタイムアウトを待たず即失敗する (`common/orch.rs::until_healthy`)。
978+
シム経由の実機: `orchestration_mixed_real` 3 件 (混成 + effort の行) と `orchestration_opencode_real` 2 件が成功、`acp_opencode_real` は 10 件中 9 件成功
979+
(無料モデルが合言葉を答えない回がある `real_opencode_system_prompt_reaches_the_agent` は不安定、再実行で通る)。残存プロセス 0。
972980
- 残課題: 上の OpenCode の環境問題 (混成の実機テストが通せない)。モデルが effort を持たなくなった古い設定はセッション起動が失敗する (警告のみで既定に戻す挙動にはしていない)。
973981
設定モーダルは項目が 1 つだけ (足す作りは用意した)。

‎crates/yhtye-core/tests/acp_opencode_real.rs‎

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,9 @@ use std::sync::{Arc, Mutex};
1212
use std::time::{Duration, Instant};
1313

1414
use async_trait::async_trait;
15-
use common::opencode::{MODEL, REAL_TIMEOUT, assert_no_new_servers, opencode_servers};
15+
use common::opencode::{
16+
MODEL, REAL_TIMEOUT, assert_no_new_servers, opencode_harness, opencode_preset, opencode_servers,
17+
};
1618
use common::{Session, assert_group_gone, message_text};
1719
use serde_json::{Value, json};
1820
use tokio::sync::mpsc;
@@ -70,7 +72,7 @@ async fn start(harness: &HarnessConfig, cwd: &Path, options: SpawnOptions) -> Oc
7072
}
7173

7274
async fn start_build(cwd: &Path, options: SpawnOptions) -> Oc {
73-
start(&HarnessConfig::opencode(MODEL), cwd, options).await
75+
start(&opencode_harness(MODEL), cwd, options).await
7476
}
7577

7678
/// Shuts down; no process of the agent's group and no `opencode serve --stdio`
@@ -432,7 +434,7 @@ async fn real_opencode_implementer_reports_through_mcp() {
432434
async fn real_opencode_model_list_comes_from_the_preset_probe() {
433435
let servers_before = opencode_servers();
434436
let dir = tempfile::tempdir().expect("tempdir");
435-
let preset = yhtye_core::agents::HarnessPreset::opencode(MODEL, Vec::new());
437+
let preset = opencode_preset(MODEL, Vec::new());
436438
let started = Instant::now();
437439
let models = yhtye_core::agents::probe_models(
438440
&preset,
@@ -468,10 +470,10 @@ async fn real_opencode_model_list_comes_from_the_preset_probe() {
468470
#[tokio::test]
469471
#[ignore = "real OpenCode (muse-spark free)"]
470472
async fn real_opencode_efforts_are_read_per_model_and_applied() {
471-
use yhtye_core::agents::{AgentRole, HarnessPreset, probe_efforts};
473+
use yhtye_core::agents::{AgentRole, probe_efforts};
472474
let servers_before = opencode_servers();
473475
let dir = tempfile::tempdir().expect("tempdir");
474-
let preset = HarnessPreset::opencode(MODEL, Vec::new());
476+
let preset = opencode_preset(MODEL, Vec::new());
475477
let started = Instant::now();
476478
let efforts = probe_efforts(
477479
&preset,
@@ -531,7 +533,7 @@ async fn real_opencode_session_load_in_another_directory_fails_fast() {
531533
..SpawnOptions::default()
532534
};
533535
let started = Instant::now();
534-
let result = spawn_agent(&HarnessConfig::opencode(MODEL), b.path(), opts, tx).await;
536+
let result = spawn_agent(&opencode_harness(MODEL), b.path(), opts, tx).await;
535537
let elapsed = started.elapsed();
536538
let err = result.err().expect("load in another directory is rejected");
537539
eprintln!("rejected after {elapsed:?}: {err}");

‎crates/yhtye-core/tests/common/mod.rs‎

Lines changed: 13 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -24,21 +24,25 @@ fn workspace_root() -> PathBuf {
2424
Path::new(env!("CARGO_MANIFEST_DIR")).join("../..")
2525
}
2626

27-
/// Path of the fake agent binary, building it once per test process.
28-
pub fn fake_agent_bin() -> PathBuf {
29-
static BIN: OnceLock<PathBuf> = OnceLock::new();
30-
BIN.get_or_init(|| {
27+
/// Path of a binary of `yhtye-fake-agent`, building the crate once per test process.
28+
pub fn fake_agent_crate_bin(name: &str) -> PathBuf {
29+
static BUILT: OnceLock<()> = OnceLock::new();
30+
BUILT.get_or_init(|| {
3131
let status = std::process::Command::new(env!("CARGO"))
3232
.args(["build", "--quiet", "-p", "yhtye-fake-agent"])
3333
.current_dir(workspace_root())
3434
.status()
3535
.expect("run cargo build for yhtye-fake-agent");
3636
assert!(status.success(), "building yhtye-fake-agent failed");
37-
let target = std::env::var_os("CARGO_TARGET_DIR")
38-
.map_or_else(|| workspace_root().join("target"), PathBuf::from);
39-
target.join("debug").join("yhtye-fake-agent")
40-
})
41-
.clone()
37+
});
38+
let target = std::env::var_os("CARGO_TARGET_DIR")
39+
.map_or_else(|| workspace_root().join("target"), PathBuf::from);
40+
target.join("debug").join(name)
41+
}
42+
43+
/// Path of the fake agent binary.
44+
pub fn fake_agent_bin() -> PathBuf {
45+
fake_agent_crate_bin("yhtye-fake-agent")
4246
}
4347

4448
/// A fake-agent harness configured like Claude Code (bypass mode, haiku model).

‎crates/yhtye-core/tests/common/opencode.rs‎

Lines changed: 37 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ use std::path::Path;
55
use std::time::Duration;
66

77
use yhtye_core::acp::{AgentEvent, HarnessConfig};
8+
use yhtye_core::agents::HarnessPreset;
89
use yhtye_core::api::{ApiEvent, ApiEventBody};
910
use yhtye_core::runtime::OrchestrationConfig;
1011

@@ -76,7 +77,41 @@ pub fn real_config(dir: &Path, orchestrator: HarnessConfig) -> OrchestrationConf
7677
config(
7778
dir,
7879
orchestrator,
79-
HarnessConfig::opencode(MODEL),
80-
HarnessConfig::opencode(MODEL),
80+
opencode_harness(MODEL),
81+
opencode_harness(MODEL),
8182
)
8283
}
84+
85+
/// Runs `opencode acp` through the test-only warm-up shim
86+
/// (`crates/yhtye-fake-agent/src/bin/opencode-warmup-shim.rs`). OpenCode 2.0.18
87+
/// snapshots the model catalog of the first `session/new` of a process before
88+
/// the providers are loaded (upstream anomalyco/opencode#50236), so `model not
89+
/// found` follows; the shim sends a throwaway `session/new` first. Yhtye itself
90+
/// does not work around it (user decision); only tests use the shim.
91+
pub fn with_warmup_shim(mut harness: HarnessConfig) -> HarnessConfig {
92+
assert_eq!(harness.command, "opencode", "an OpenCode harness");
93+
harness.command = super::fake_agent_crate_bin("opencode-warmup-shim")
94+
.display()
95+
.to_string();
96+
harness
97+
}
98+
99+
/// [`HarnessConfig::opencode`] behind the warm-up shim.
100+
pub fn opencode_harness(model: &str) -> HarnessConfig {
101+
with_warmup_shim(HarnessConfig::opencode(model))
102+
}
103+
104+
/// [`HarnessPreset::opencode`] (every role and the probe) behind the warm-up shim.
105+
pub fn opencode_preset(fallback_model: &str, env_remove: Vec<String>) -> HarnessPreset {
106+
let mut p = HarnessPreset::opencode(fallback_model, env_remove);
107+
for h in [
108+
&mut p.orchestrator,
109+
&mut p.implementer,
110+
&mut p.investigator,
111+
&mut p.reviewer,
112+
] {
113+
*h = with_warmup_shim(h.clone());
114+
}
115+
p.probe = p.probe.map(with_warmup_shim);
116+
p
117+
}

‎crates/yhtye-core/tests/common/orch.rs‎

Lines changed: 28 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ use tokio::sync::mpsc;
88
use yhtye_core::acp::{AgentEvent, AgentOutput, HarnessConfig};
99
use yhtye_core::agents::AgentCatalog;
1010
use yhtye_core::api::{ApiEvent, ApiEventBody};
11-
use yhtye_core::domain::{DomainConfig, DomainEvent, TaskStatus};
11+
use yhtye_core::domain::{DomainConfig, DomainEvent, HelpKind, TaskStatus};
1212
use yhtye_core::git::NoopGit;
1313
use yhtye_core::runtime::{Orchestration, OrchestrationConfig};
1414

@@ -79,6 +79,33 @@ pub async fn until(
7979
}
8080
}
8181

82+
/// Like [`until`] for real-agent runs, which never expect a failure: a session
83+
/// that fails to start (`SessionFailed`) or an agent that crashed
84+
/// (`AgentCrashed` help) fails the test at once with the error instead of after
85+
/// the whole timeout.
86+
pub async fn until_healthy(
87+
rx: &mut mpsc::UnboundedReceiver<ApiEvent>,
88+
seen: &mut Vec<ApiEvent>,
89+
timeout: Duration,
90+
done: impl Fn(&ApiEvent) -> bool,
91+
) {
92+
until(rx, seen, timeout, |ev| {
93+
match &ev.body {
94+
ApiEventBody::SessionFailed { session, error } => {
95+
panic!("{session} failed to start: {error}")
96+
}
97+
ApiEventBody::Domain {
98+
event: DomainEvent::HelpRaised { help },
99+
} if help.kind == HelpKind::AgentCrashed => {
100+
panic!("agent crashed (task {}): {}", help.task, help.message)
101+
}
102+
_ => {}
103+
}
104+
done(ev)
105+
})
106+
.await;
107+
}
108+
82109
pub fn message<'a>(ev: &'a ApiEvent, session: &str) -> Option<&'a str> {
83110
match &ev.body {
84111
ApiEventBody::Agent {

‎crates/yhtye-core/tests/orchestration_mixed_real.rs‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -9,8 +9,8 @@ mod common;
99
use std::sync::Arc;
1010
use std::time::Duration;
1111

12-
use common::opencode::{MODEL as FREE, assert_no_new_servers, opencode_servers};
13-
use common::orch::{shutdown_and_check, tool_calls, until};
12+
use common::opencode::{MODEL as FREE, assert_no_new_servers, opencode_preset, opencode_servers};
13+
use common::orch::{shutdown_and_check, tool_calls, until_healthy};
1414
use common::real::{MODEL as HAIKU, REAL_TIMEOUT, is_orchestrator_turn_end, real_git_config};
1515
use common::repo::TempRepo;
1616
use yhtye_core::acp::{AgentEvent, AgentOutput};
@@ -35,7 +35,7 @@ fn opencode() -> AgentChoice {
3535
fn catalog(roles: &[(AgentRole, AgentChoice)]) -> AgentCatalog {
3636
let presets = vec![
3737
HarnessPreset::claude_code(HAIKU),
38-
HarnessPreset::opencode(
38+
opencode_preset(
3939
OPENCODE_FALLBACK_MODEL,
4040
inherited_opencode_env_remove(|k| std::env::var_os(k)),
4141
),
@@ -151,8 +151,8 @@ async fn run_with(
151151
.unwrap_or_else(|e| panic!("{e}"));
152152
let mut events = Vec::new();
153153
orch.send_user_message(request).expect("send");
154-
until(&mut rx, &mut events, REAL_TIMEOUT, group_merged).await;
155-
until(&mut rx, &mut events, REAL_TIMEOUT, is_orchestrator_turn_end).await;
154+
until_healthy(&mut rx, &mut events, REAL_TIMEOUT, group_merged).await;
155+
until_healthy(&mut rx, &mut events, REAL_TIMEOUT, is_orchestrator_turn_end).await;
156156
eprintln!("tool calls: {:#?}", tool_calls(&events));
157157
(orch, events)
158158
}

‎crates/yhtye-core/tests/orchestration_opencode_real.rs‎

Lines changed: 12 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -5,13 +5,14 @@
55
mod common;
66

77
use common::opencode::{
8-
MODEL, REAL_TIMEOUT, assert_model, assert_no_new_servers, opencode_servers, real_config,
8+
MODEL, REAL_TIMEOUT, assert_model, assert_no_new_servers, opencode_harness, opencode_servers,
9+
real_config,
910
};
10-
use common::orch::{prompts_to, shutdown_and_check, summary, tool_calls, until};
11+
use common::orch::{prompts_to, shutdown_and_check, summary, tool_calls, until_healthy};
1112
use common::real::is_orchestrator_turn_end;
1213
use std::time::Duration;
1314
use tokio::sync::mpsc;
14-
use yhtye_core::acp::{AgentEvent, AgentOutput, HarnessConfig};
15+
use yhtye_core::acp::{AgentEvent, AgentOutput};
1516
use yhtye_core::api::{ApiEvent, ApiEventBody};
1617
use yhtye_core::domain::Role;
1718
use yhtye_core::runtime::{ORCHESTRATOR_SESSION, Orchestration};
@@ -38,12 +39,12 @@ async fn run_request(
3839
) -> Vec<ApiEvent> {
3940
let mut events = Vec::new();
4041
orch.send_user_message(request).expect("send");
41-
until(rx, &mut events, REAL_TIMEOUT, |e| {
42+
until_healthy(rx, &mut events, REAL_TIMEOUT, |e| {
4243
matches!(&e.body, ApiEventBody::Prompted { session, text }
4344
if session == ORCHESTRATOR_SESSION && text.contains("[yhtye:group_settled]"))
4445
})
4546
.await;
46-
until(rx, &mut events, REAL_TIMEOUT, is_orchestrator_turn_end).await;
47+
until_healthy(rx, &mut events, REAL_TIMEOUT, is_orchestrator_turn_end).await;
4748
events
4849
}
4950

@@ -56,10 +57,9 @@ async fn real_opencode_orchestrator_creates_task_and_sub_agent_reports_back() {
5657
let dir = tempfile::tempdir().expect("tempdir");
5758
let readme = dir.path().join("README.md");
5859
std::fs::write(&readme, "# demo\n").expect("write README");
59-
let (orch, mut rx) =
60-
Orchestration::start(real_config(dir.path(), HarnessConfig::opencode(MODEL)))
61-
.await
62-
.unwrap_or_else(|e| panic!("{e}"));
60+
let (orch, mut rx) = Orchestration::start(real_config(dir.path(), opencode_harness(MODEL)))
61+
.await
62+
.unwrap_or_else(|e| panic!("{e}"));
6363
let events = run_request(
6464
&orch,
6565
&mut rx,
@@ -121,10 +121,9 @@ async fn real_opencode_implement_then_review() {
121121
let dir = tempfile::tempdir().expect("tempdir");
122122
let readme = dir.path().join("README.md");
123123
std::fs::write(&readme, "# demo\n").expect("write README");
124-
let (orch, mut rx) =
125-
Orchestration::start(real_config(dir.path(), HarnessConfig::opencode(MODEL)))
126-
.await
127-
.unwrap_or_else(|e| panic!("{e}"));
124+
let (orch, mut rx) = Orchestration::start(real_config(dir.path(), opencode_harness(MODEL)))
125+
.await
126+
.unwrap_or_else(|e| panic!("{e}"));
128127
let events = run_request(
129128
&orch,
130129
&mut rx,

‎crates/yhtye-fake-agent/Cargo.toml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ publish = false
88

99
[dependencies]
1010
agent-client-protocol.workspace = true
11-
tokio.workspace = true
11+
tokio = { workspace = true, features = ["io-std"] }
1212
serde.workspace = true
1313
serde_json.workspace = true
1414
rmcp = { workspace = true, features = ["client", "transport-streamable-http-client-reqwest"] }

0 commit comments

Comments
 (0)