opencode RL literal bridge: rollout_details correlation + HARBOR_MODEL_ENDPOINT baseURL#7
Merged
Merged
Conversation
added 5 commits
July 16, 2026 07:02
…ollout_details OpenCode is a CLI agent that talks to vLLM over its own transport, bypassing harbor Chat, so context.rollout_details is never populated. The RecordProxy literal.jsonl was already parsed + merged into trajectory.json but not into rollout_details, which the RL rollout loop (MarinSkyRL) consumes for TIS logprob extraction. Add a flag-gated (collect_rollout_details, default False) bridge that also builds a RolloutDetail from the parsed literal per-turn pairs and sets context.rollout_details in populate_context_post_run. Mirrors Chat.rollout_details + the per-turn length-parity guard (8737426): a turn whose logprobs length != completion_token_ids length (or absent) records an empty logprob list so downstream TIS flags it rather than mis-pairing. Flag off => byte-identical (rollout_details stays None); non-RL/datagen paths unaffected. Adds S0 repro (rollout_details None despite non-empty literal log) and S1 flag-on parity tests.
…iteral capture RL runs many opencode trials concurrently against one served vLLM behind a single shared RecordProxy, so their captured completions interleave in one log. Content correlation (datagen's literal_correlator) cannot disambiguate GRPO groups (n>1 rollouts of the IDENTICAL prompt -> every continuation ambiguous). Add a per-trial correlation id transported as an HTTP header (confirmed to forward through opencode-ai 1.18.2 @ai-sdk/openai-compatible options.headers): - opencode.py mints a per-trial id when collect_rollout_details is on, stamps it into the vllm provider options.headers[x-ot-trial-id], and surfaces it on context.metadata[rollout_correlation_id] for the RL-loop glue. - RecordProxy captures the x-ot-trial-id request header onto each JSONL entry (trial_id) on both the streaming and non-streaming paths. - new harbor.literal.rollout_build: build_rollout_details_from_pairs (the shared Chat.rollout_details-mirroring parity builder, extracted from opencode.py) and build_rollout_details_for_trial (filter a shared log by trial_id, order by arrival -> one RolloutDetail; spans opencode auto-compaction boundaries). Flag off => no id, no header, no metadata, trial_id=None (byte-identical; datagen/content path untouched). Tests: n>1 identical-seed no-bleed, compaction ordering, header round-trip, parity guard. 83 harbor unit tests green.
…teral-bridge header The per-trial correlation-id header forwards through @ai-sdk/openai-compatible on opencode-ai 1.18.2 (older 1.0.x dropped provider options, opencode #5674). RL runs relying on the literal bridge must pin version 1.18.2 in the agent config. Comment only; no behavior change.
…er (RL literal bridge GAP C) The RL rollout generator forces model_name="hosted_vllm/<alias>" (terminal_bench_generator), but the baseURL/apiKey/@ai-sdk/openai-compatible + x-ot-trial-id correlation-header branch was gated on provider=="vllm" only, so it never fired for opencode-in-RL: opencode got no baseURL and no per-trial header, the shared RecordProxy could not attribute its log entries, and correlation no-op'd. Treat "hosted_vllm" like "vllm" in the config-writer AND the two downstream env-selection gates in run() (OPENAI_API_KEY/OPENAI_BASE_URL keys + the OPENCODE_DUMMY_KEY injection). Adds real-path tests over the actual config-writer under model_name="hosted_vllm/..." (header + npm + baseURL), byte-identical flag-off, and version/opencode_config reaching the ctor. The prior header test used "vllm/..." so it passed while the real path no-op'd.
…(not OPENAI_BASE_URL) _build_register_config_command sourced the self-hosted vLLM baseURL from OPENAI_BASE_URL. That var is RESERVED for genuine OpenAI traffic — the LLM-judge verifiers on the worker read it — so having the RL runner publish the vLLM capability URL there silently misroutes every judge call to vLLM. Prefer the harbor-specific HARBOR_MODEL_ENDPOINT (the var the RL runner already publishes for the co-located served model); keep OPENAI_BASE_URL as a back-compat fallback so nothing else regresses. Also forward HARBOR_MODEL_ENDPOINT into the sandbox env for the vllm/openai providers, symmetric with the baseURL source (non-load-bearing: the baseURL is baked into opencode.json). Adds 2 tests (prefers HARBOR_MODEL_ENDPOINT; falls back to OPENAI_BASE_URL); 51 opencode tests pass.
penfever
pushed a commit
that referenced
this pull request
Jul 18, 2026
…indow Port of the runtime fix from harbor main 826c2e4 (PR #11) onto penfever/working (the branch the TACC eval/datagen editable clones track; main's version sits atop opencode PRs #7/#8 which working lacks). opencode defaults a custom (openai-compatible) provider's max_tokens to ~32000 (opencode harbor-framework#1735); _build_register_config_command wrote an EMPTY per-model dict, so against a 32768 served window every real prompt 400s ('maximum context length ... you requested 32000 output tokens') -> opencode exits 1 -> NonZeroAgentExitCodeError -> reward 0 on 100% of trials. Writes a bounded per-model limit {context,output} into opencode.json: output = model_info.max_output_tokens or 4096; window = model_info.max_input_tokens or 32768; context = window - output, so prompt+output <= window always. model_info is the ctor kwarg the eval path passes (eval_opencode_ctx32k.yaml: max_input 32768 / max_output 16384 -> limit {context:16384, output:16384}); safe class-default fallback otherwise. Keeps working's env-based baseURL (OPENAI_BASE_URL), which the eval sbatch sets. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_0191M6UMWFw2eWr5y4GwC1Qg
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The harbor half of the opencode RL literal bridge.
Local: 51 opencode tests + 9 rollout_build tests pass; ruff + IDE diagnostics clean.