[NA] [SDK] fix: preserve StartSpanParameters when merging opik_args by pragnyanramtha · Pull Request #6735 · comet-ml/opik

pragnyanramtha · 2026-05-17T03:38:58Z

Details

Preserve all existing StartSpanParameters fields when span-level opik_args are merged into tracked-function start parameters.

The affected helper should only merge span tags and metadata. It previously rebuilt the dataclass manually and copied a fixed subset of fields, which could drop fields that were already computed from the trace/decorator configuration, such as environment and thread_id.

This is not intended to let a child span choose a different environment from its trace. It preserves the trace/decorator-selected environment while applying unrelated span-level opik_args fields.

Change checklist

User facing
Documentation update

Issues

Fixes opik_args span merge drops StartSpanParameters fields #6761
OPIK-NA

AI-WATERMARK

AI-WATERMARK: yes

If yes:
- Tools: Codex CLI
- Model(s): GPT-5
- Scope: focused Python SDK bug fix and regression tests
- Human verification: local pytest and ruff commands listed below

Testing

. .venv/bin/activate && pytest sdks/python/tests/unit/decorator/test_opik_args.py::TestPresetTraceId::test_track__environment_with_span_opik_args__environment_is_preserved sdks/python/tests/unit/decorator/test_opik_args.py::TestOpikArgs::test_apply_opik_args_to_start_span_params__with_span_args -q -> 2 passed in 0.79s
. .venv/bin/activate && pytest sdks/python/tests/unit/decorator/test_opik_args.py -q -> 29 passed in 1.61s
uvx ruff check sdks/python/src/opik/decorator/opik_args/helpers.py sdks/python/tests/unit/decorator/test_opik_args.py -> All checks passed!
uvx ruff format --check sdks/python/src/opik/decorator/opik_args/helpers.py sdks/python/tests/unit/decorator/test_opik_args.py -> 2 files already formatted

Scenarios validated: helper field preservation, decorator-level environment propagation when span opik_args are supplied, and existing opik_args behavior in the affected unit file.

Not run: full Python SDK e2e suite; this change is isolated to decorator argument merging and is covered by unit tests.

Documentation

No documentation update. This restores existing documented @track(environment=...) behavior when combined with opik_args without changing span/trace environment selection semantics.

Copilot

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

petrotiurin

Thank you for opening a pull request, @pragnyanramtha! Environment is generally supposed to be driven by the trace, with all the spans resulting from it inheriting the same value. This is to avoid a misconfiguration when a span is logged into a different environment from its trace and resulting in spans missing from the trace view in Opik.

Is there a specific problem that you're trying to resolve with this PR? Could you file an issue in this repo describing it?

pragnyanramtha · 2026-05-18T18:08:31Z

Thanks for the review. I filed the issue here: #6761

The specific problem is narrower than letting a span choose a different environment from its trace. When span-level opik_args are supplied, apply_opik_args_to_start_span_params reconstructs StartSpanParameters by hand and only copies a subset of fields. That can drop already-computed fields such as environment and thread_id while applying unrelated tag/metadata overrides.

This PR preserves the parameters that were already selected by the trace/decorator config and only replaces the merged tags/metadata. If you prefer, I can also narrow the regression wording so it is explicitly about preserving the trace-derived environment rather than span-level environment override behavior.

pragnyanramtha · 2026-05-18T20:03:57Z

I updated the PR description to narrow the scope around the review concern: this is not meant to let a child span choose a different environment from its trace. The fix preserves fields that were already computed from trace/decorator configuration, such as environment and thread_id, while only applying span-level tags/metadata from opik_args.

Issue #6761 has the same narrowed reproduction/expected behavior. Could you re-check when you have a chance?

…reserve-opik-args-environment

…ment

petrotiurin

Thanks for clarifying the purpose of this PR. Appreciate you taking the time to put a fix together!
Tests are all green - this is good to be merged.

* initial local POC for visual comparison tests * stabilize existing tests * adding allure * improve clean up * add mask for demo data * [OPIK-6505] [BE] Split stats query and gate legacy feedback_scores UNION (#6713) * [OPIK-6505] [BE] Split traces/spans stats query and gate legacy feedback_scores UNION Splits the project-stats endpoints into two parallel queries (traces+spans aggregation and feedback-score aggregation) and runs them via Mono.zip. The per-trace/per-span JOIN against the rich groupArray-tuple feedback CTE was the dominant cost at scale. Adds a `workspaces.has_legacy_scores` flag (defaults TRUE) detected during workspace version determination; the stats SQL gates the legacy feedback_scores table UNION on it so workspaces with no legacy data skip the empty-table scan. Applies the same split + gate to SpanDAO.getStats. On the OPIK-6505 customer dataset this brings end-to-end stats wall clock down from ~78s to ~4s unfiltered and ~0.3s with a trace-side filter, with byte-identical API output (144-key feedback map matched to float64 precision). * [OPIK-6505] [BE] Align StatsMapperTest with split-A / split-B contract The split moved feedback-score row mapping out of mapProjectStats (split-A no longer carries those columns) into a new mapProjectScoresStats(Row) used by SELECT_FEEDBACK_SCORES_STATS / SELECT_SPAN_FEEDBACK_SCORES_STATS. Update the test to assert the new contract: - mapProjectScoresStats_withSpanFeedbackScores_returnsSpanFeedbackScoresStats asserts the new method emits the feedback stat items. - mapProjectStats_doesNotEmitFeedbackScoreStats asserts the trace+spans mapper now intentionally drops feedback-score columns. * [OPIK-6505] [BE] Address review: NPE guard, consolidate hasLegacyScores, add regression test - StatsMerger.merge: avoid NPE for projects with no feedback rows by falling back to ProjectStats.empty() at the call site instead of guarding inside the single-project overload. - Move the reactive has_legacy_scores lookup onto WorkspacesService (Mono<Boolean> hasLegacyScores) so TraceDAO and SpanDAO share one implementation instead of duplicating a Schedulers.boundedElastic wrapper. - Add ProjectsResourceTest coverage that flips the workspace flag and re-hits /stats to verify the endpoint stays consistent. * [OPIK-6505] [BE] Fix CI regressions: thread stats feedback + merge order - StatsMapper.mapProjectStats: re-emit feedback_scores when the row carries that column. Thread stats inline feedback in the same row and call this mapper directly with no merge step; the prior linter refactor dropped them silently. Split-A trace/span rows omit the column, so the metadata-guard is a no-op there. - StatsMerger.merge: splice the feedback ProjectStats into the base list at the canonical position (just before guardrails / error-count entries) instead of appending. The pre-split mapper emitted feedback there, and StatsUtils-built expected stats assert that order. - StatsMapperTest: replace the "drops feedback" assertion with two cases — emits when the column is present, omits when it is not — matching the real contract for split-A vs thread paths. * [OPIK-6505] [BE] Defend stats Mono.zip against empty traces/spans branch SELECT_TRACES_SPANS_STATS and SELECT_SPANS_STATS both end with GROUP BY project_id, so when no rows match the filter the projection emits zero rows and singleOrEmpty() completes empty. Mono.zip then drops the feedback branch as well, so the response silently loses feedback-only stats. Match the safety net already on the feedback side by attaching .switchIfEmpty(Mono.just(new ProjectStats(List.of()))) to both traces/spans branches. * [OPIK-6505] [BE] Address review: gate legacy probe, dedupe stats orchestration WorkspaceVersionService.persistAndEmitBlocking now reuses the existing findById(workspaceId) lookup (already loaded for lastKnownVersion) to gate the ClickHouse probe + MySQL upsert behind the stored has_legacy_scores flag. Once that flag is false the legacy feedback_scores table only ever shrinks (no new writes land there), so the per-cache-miss probe is wasted work. When the flag is true we still probe but only upsert when the result differs, eliminating the redundant rewrites flagged in review. Extracted the split-A / split-B Mono.zip orchestration into StatsMerger.zipAndMerge so SpanDAO.getStats and TraceDAO.getStats invoke the same logic. The helper centralises the empty-default safety net that the prior commit introduced ad-hoc on each side — future tweaks to the zip semantics now live in one place. * [OPIK-6505] [BE] Address review: short-circuit zipAndMerge, trim comments zipAndMerge: when the aggregates Mono emits empty, return empty stats instead of forcing a feedback-only response. This matches the traces-driven scope the map overload enforces (no resurrecting projects from feedback alone). Use the existing ProjectStats.empty() factory instead of allocating new ProjectStats(List.of()). Trim verbose comments on StatsMerger, mapProjectStats, and the WorkspaceVersionService gated probe down to one-line whys. * [OPIK-6505] [BE] Add regression test with real legacy_scores data Seeds a project via buildProjectStats then writes one feedback score directly into the legacy `feedback_scores` ClickHouse table via the DAO's author=null path (the public API always routes to authored_feedback_scores). Asserts the /stats response — full object recursive comparison — includes the legacy score when has_legacy_scores is true and drops it after flipping the workspace flag to false. Closes the gap @ldaugusto flagged on the PR: previously the UNION branch was only verified by manual production benchmarks. * [OPIK-6505] [BE] Cover all three legacy-UNION endpoints with real-data tests Each test writes one feedback score directly into the legacy feedback_scores ClickHouse table via the author=null DAO path (the public API always routes to authored_feedback_scores), then triggers the natural workspace-version determination flow via workspaceResourceClient.getWorkspaceVersion so the probe sees the legacy rows and persists has_legacy_scores=true the same way it does in production. Stats assertions use whole-object recursive comparison. - ProjectsResourceTest: multi-project /projects/stats path. - MultiValueFeedbackScoresE2ETest: single-project /traces/stats and /spans/stats paths (new SQL templates on the span side). Drops the previous flag=false branch on the multi-project test — that state is impossible when legacy data exists, the existing getProjects__whenHasLegacyScoresFlipped__thenStatsStayConsistent test already covers the no-legacy-data case. * [NA] [SDK] [DOCS] Update automatically OpenAPI spec and Fern code (#6734) Co-authored-by: Andres Cruz <[email protected]> * [NA] [BE] Update model prices file (#6736) Co-authored-by: Andres Cruz <[email protected]> * [NA] [BE][FE] chore: sync provider model definitions (#6737) Co-authored-by: Andres Cruz <[email protected]> * build(deps): bump dev.langchain4j:langchain4j-bom in /apps/opik-backend (#6738) Bumps [dev.langchain4j:langchain4j-bom](https://github.com/langchain4j/langchain4j) from 1.14.0 to 1.15.0. - [Release notes](https://github.com/langchain4j/langchain4j/releases) - [Commits](https://github.com/langchain4j/langchain4j/compare/1.14.0...1.15.0) --- updated-dependencies: - dependency-name: dev.langchain4j:langchain4j-bom dependency-version: 1.15.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Andres Cruz <[email protected]> * build(deps-dev): bump com.diffplug.spotless:spotless-maven-plugin (#6739) Bumps [com.diffplug.spotless:spotless-maven-plugin](https://github.com/diffplug/spotless) from 3.4.0 to 3.5.1. - [Release notes](https://github.com/diffplug/spotless/releases) - [Changelog](https://github.com/diffplug/spotless/blob/main/CHANGES.md) - [Commits](https://github.com/diffplug/spotless/compare/maven/3.4.0...maven/3.5.1) --- updated-dependencies: - dependency-name: com.diffplug.spotless:spotless-maven-plugin dependency-version: 3.5.1 dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Andres Cruz <[email protected]> * build(deps): bump org.redisson:redisson in /apps/opik-backend (#6740) Bumps [org.redisson:redisson](https://github.com/redisson/redisson) from 4.3.1 to 4.4.0. - [Release notes](https://github.com/redisson/redisson/releases) - [Changelog](https://github.com/redisson/redisson/blob/master/CHANGELOG.md) - [Commits](https://github.com/redisson/redisson/compare/redisson-4.3.1...redisson-4.4.0) --- updated-dependencies: - dependency-name: org.redisson:redisson dependency-version: 4.4.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Andres Cruz <[email protected]> * [INFRA] feat: respect pre-set port env vars when initializing worktree ports (#6558) init_worktree_ports() unconditionally overwrote NGINX_PORT (and every sibling port) with BASE + PORT_OFFSET, so callers couldn't pin a single port via env without also disabling the worktree offset for the rest. Switch each assignment to the ${VAR:-default} idiom so pre-set values win, fix the docker-compose default for OPIK_REVERSE_PROXY_URL to follow NGINX_PORT, and document the knob in the docker-compose README. Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]> Co-authored-by: Andres Cruz <[email protected]> * [OPIK-6544] [FE] feat: add edit option to dataset item row menu (#6741) Wires setActiveRowId into the row-actions cell via customMeta and adds an Edit menu item above Delete (with a separator). Works for both regular dataset items and test suite items since both share the DatasetItemsTab and DatasetItemRowActionsCell wiring. Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]> * [NA] [QA] fix: wait for traces table to fully render before scanning Moderation header (#6743) The online-scoring moderation flow test polled `table thead th` with a fixed 3s wait per reload. On production the traces table renders static columns first and appends dynamic feedback-score columns after a second API call, so the scan ran before the Moderation header was mounted and every retry re-raced the same too-short timer. All six provider/model variants were failing identically only on the prod sanity launches while local, staging, self-hosted, and post-merge stayed green. Switched each retry to use Playwright auto-retrying assertions: `toHaveCount` on `tbody tr` to confirm the rows are present, then `toBeVisible` on the Moderation header before reading the column index. Per-attempt budget is bounded so the 25-attempt loop still fits in the 120s test timeout in the happy path. * docs(changelog): weekly changelog for 2026-05-19 (#6742) * docs(changelog): weekly changelog for 2026-05-19 Covers releases 2.0.32–2.0.37. Highlights: - Python/TS SDK client-side prompt caching with trace metadata injection - New `opik migrate dataset` CLI command - opik connect CLI improvements (Rich errors, auto-configure, instant disconnect) - Environments: auto-assigned colors, inline validation errors, SDK field preservation - Playground: suppress Gemma 4 reasoning traces, updated Gemini/Vertex AI defaults - Google ADK integration re-patching fix - Bug fixes: silent dataset item loss, self-hosted onboarding loop, CSV upload visibility - Performance: dataset streaming, workspace selector load time https://claude.ai/code/session_01T9yZkbs2iJSNzHYEFGr4ms * docs(changelog): remove opik migrate dataset section (to be documented separately) https://claude.ai/code/session_01T9yZkbs2iJSNzHYEFGr4ms * docs(changelog): remove introductory boilerplate line https://claude.ai/code/session_01T9yZkbs2iJSNzHYEFGr4ms * docs(changelog): move environment items to bug fixes section Environment feature was already announced in 2026-05-12; these are follow-up fixes and improvements, not a new feature introduction. https://claude.ai/code/session_01T9yZkbs2iJSNzHYEFGr4ms * docs(changelog): consolidate Playground and ADK sections into bug fixes Keep prompt caching and opik connect as the two featured sections. Playground fixes, ADK re-patching fix, and all other items are now under Bug Fixes & Improvements. https://claude.ai/code/session_01T9yZkbs2iJSNzHYEFGr4ms --------- Co-authored-by: Claude <[email protected]> * [OPIK-6417] [SDK] feat: cascade trace + span comments in opik migrate dataset (#6714) * [OPIK-6417] [SDK] feat: cascade trace + span comments in opik migrate dataset Slice 4 of `opik migrate dataset`. The Slice 3 cascade already copies traces + spans + feedback scores + assertion results, but comments live as `READ_ONLY` on the trace/span Public view -- they can't ride along `create_traces` / `create_spans` and were silently dropped at the destination. Re-emit them via the dedicated single-comment write endpoints (`POST /v1/private/traces/{id}/comments`, `POST /v1/private/spans/{id}/comments`) after the destination trace/span flushes. Each POST is wrapped with the existing `ensure_rest_api_call_respecting_rate_limit` helper; order is preserved by iterating the source `comments` list in place so the destination read order matches. Counters surface on a new `cascade_experiments_summary` audit record alongside `traces_migrated` / `spans_migrated` (additive -- the existing `cascade_experiments` action record shape is unchanged). Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * test(e2e): use verifiers for comment round-trip assertions Extend `verifiers.verify_trace` / `verify_span` with an optional `comments` kwarg that asserts equality on the source-ordered list of comment texts. Refactor the cascade-comments e2e to delegate to the shared verifiers instead of hand-rolling REST polling and content/order checks. Addresses PR review feedback from baz-reviewer[bot] on PR #6714: comment text/order is part of the `TracePublic.comments` / `SpanPublic.comments` contract, so it belongs on the shared verifier surface (consistent with how the sibling `test_migrate_dataset__evaluate_shape__round_trips` test already verifies traces via `verifiers.verify_trace`). Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]> * Update versions to 2.0.38 and bump base version to 2.0.39 * [OPIK-6482] [BE] fix: add (workspace_id, scope, id) index on dashboards to avoid filesort OOM (#6733) * [OPIK-6482] [BE] fix: add (workspace_id, scope, id) index on dashboards to satisfy ORDER BY from index and avoid filesort OOM The list-dashboards query SELECT * FROM dashboards WHERE workspace_id = ? AND scope = ? ORDER BY id DESC LIMIT ? OFFSET ? falls back to filesort because no index covers both the equality filter and the ORDER BY key. With SELECT * including the JSON `config` column, the filesort step copies the full row into the sort buffer, and Aurora MySQL 8.0 raises ER_OUT_OF_SORTMEMORY (errno 1038) even at LIMIT=1 due to JSON addon-field sizing in the sort buffer. Observed in Netflix opik state DB: 168 failed calls between 2026-05-02 and 2026-05-12, surfacing as 500s for the affected workspace. Adding (workspace_id, scope, id) lets the optimizer satisfy the equality + ORDER BY id DESC from index order via a backward index scan, eliminating the filesort step entirely. The JSON column is still selected but is never copied into a sort buffer. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * [OPIK-6482] [BE] address review: bump migration to 000070 and add trailing blank line - Bump 000069 -> 000070 (000069 was taken by add_has_legacy_scores_to_workspaces after this branch was opened) - Add trailing blank line per Liquibase migration convention Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]> * [OPIK-6511] [FE] fix: hide Assistant sidebar wrapper when ollieEnabled toggle is off (#6725) * [OPIK-3662] [INFRA] feat: add actionlint workflow validation (#6600) * [OPIK-3662] [INFRA] feat: add actionlint workflow validation Adds actionlint as a CI gate for changed GitHub Actions workflow files, plus a matching pre-commit hook with graceful degradation when the binary is not installed locally. Both run with `SHELLCHECK_OPTS=--severity=warning` so the pre-existing SC2086 backlog (tracked in OPIK-6323) does not gate PRs while real warning+ findings still do. Real defects fixed alongside the enabling change: - 9 outdated action versions (publish_cursor_extension.yml@v2, setup-python@v3/v4 in 4 files, setup-node@v3 in 3 files, cache@v3, release-drafter@v5) - Script-injection risk in trigger_test_env_on_label.yaml: pull_request.head.ref is now passed via env: instead of interpolated into actions/github-script body - Deprecated set-output -> $GITHUB_OUTPUT (documentation_cookbook_tests.yml) - SC2046 unquoted command substitution in same file - workflow_call default value on a required input (build_and_publish_sdk.yaml) - Undeclared workflow_dispatch input ALLURE_JOB_RUN_ID (test_docs_links.yml) Deferred to OPIK-6323: 168 SC2086 quoting cleanup, 9 remaining cosmetic events default removals, 2 remaining expression issues, and tightening the gate to scan all files at default shellcheck severity. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * fix(ci): use unicode octopus in workflow name `:octocat:` is a GitHub markdown shortcode and only renders in places that go through GFM (PR descriptions, comments). It does not render in the Actions UI workflow name, where the raw string `:octocat: Lint Workflows` was showing instead. Switching to the unicode octopus 🐙 which the Actions UI renders correctly. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * fix(ci): pin actionlint install via gh release + attestation verify Addresses Baz comment on PR #6600: the previous `bash <(curl ...)` install pulled the download script from a mutable URL with no integrity check. Replaces it with the upstream-recommended secure install path from https://github.com/rhysd/actionlint/blob/main/docs/install.md: 1. Resolve the latest release tag via `gh release view` 2. Download the linux_amd64 tarball as an immutable release asset 3. Verify its GitHub artifact attestation (Sigstore-signed by rhysd's release CI) before executing — verifies provenance, not just bytes 4. Echo the version in the workflow log for traceability This is strictly stronger than the SHA256-pinning Baz suggested: attestation verification confirms the artifact was built by the official rhysd/actionlint release workflow, not just that the bytes match a hardcoded hash. `gh` is preinstalled on GitHub-hosted runners, so no extra setup needed. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * feat(ci): also lint workflows that use changed composite actions Composite actions in .github/actions/*/action.yml can't be linted by actionlint directly (it treats them as workflows and complains about missing on:/jobs:). They're validated transitively when a workflow that references them is linted. Extends the gate (CI + pre-commit) to detect when a composite action is changed without a workflow edit, find the workflows that reference it via `uses: ./.github/actions/<name>`, and add those to the lint set. Avoids the full-tree fallback that would have tripped on the pre-existing baseline findings deferred to OPIK-6323. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * refactor(ci): drop unnecessary const layer in trigger_test_env_on_label Addresses liyaka's review comment on PR #6600: the intermediate `const headRef = process.env.HEAD_REF` and `const baseVersion = ...` lines added ceremony for values used once. Inlines `${process.env.HEAD_REF}` and `${process.env.BASE_VERSION}` directly in the template literal. The `env:` block is preserved — that's the actual security primitive. Passing `${{ github.event.pull_request.head.ref }}` via env: (rather than inlining it into the script body) prevents script injection, because env: values are read at runtime via process.env rather than substituted into the script source by `${{ }}` expansion. Added a comment block above the env: keys explaining this so the indirection is no longer mysterious. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]> * Update versions to 2.0.39 and bump base version to 2.0.40 * [OPIK-6580] [BE] [CI] fix: clear actionlint events and expression findings (#6747) Removes 10 dead `default:` values on `workflow_call` inputs that are also marked `required: true` (actionlint [events]) across 7 workflows, and resolves the two remaining [expression] findings: - quickstart_guide_snippets_test.yml: declare ALLURE_JOB_RUN_ID as a workflow_dispatch input so `github.event.inputs.ALLURE_JOB_RUN_ID` is defined. - release-wrapper-release-n-deploy.yml: drop the debug `echo "url: ${{ inputs }}"` that triggered whole-object eval (next line already echoes toJSON(inputs)). Parent: OPIK-6323. First of 5 subtasks; no SC2086 quoting work here. Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]> * [OPIK-6516] [SDK] feat(runner): add `opik connect/endpoint stop` to cleanly terminate local runners (#6730) * [NA] [SDK] refactor: group opik connect/endpoint into cli/local_runner with Rich error UX * Move connect.py, endpoint.py, _run.py, pairing.py, error_view.py under cli/local_runner/ so the pairing flow is one cohesive subpackage. * Banner now shows Workspace alongside Opik URL and Project, with values rendered bold so they read before the pairing link. * Replace the inline error string with a Rich-rendered labelled block (Reason / Workspace / URL / Config / Fix / Docs / Run) via a new RichClickError + build_config_error_block factory. Plain text stays on .message for Sentry/tests; Rich output is used in .show(). * Detect "no config file at ~/.opik.config" and surface a clear "run `opik configure`" call-to-action on every failure path. * Drop default values on internal kwargs that production callers always supply (workspace, base_url, config_file_exists, create_if_missing, etc.) so signatures reflect actual usage; add test helpers to keep call sites readable. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * [NA] [SDK][FE] feat: auto-configure on connect/endpoint + pairing context UX * opik connect/endpoint auto-launch `opik configure` when no config file exists. Skipped on --non-interactive, --headless, no TTY, or when an API key is already supplied via --api-key / OPIK_API_KEY. * Extract preflight helpers (should_create_project, maybe_auto_configure) into cli/local_runner/preflight.py so tests can target a public surface. * Pair URL now carries `&url=<ui-url>` with the `/api` suffix stripped, so the pairing page can show the user-facing instance address. * Pairing error screens render a labelled context card under the subtitle: "Workspace: X / Pairing with Opik at: <url>" — URL is a clickable link that opens the Opik UI in a new tab. * scripts/dev-runner.{sh,ps1} export TOGGLE_FORCE_WORKSPACE_VERSION=version_2 by default so a fresh local backend doesn't trip the "Workspace upgrade required" pairing screen. Override via TOGGLE_FORCE_WORKSPACE_VERSION=disabled. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * [NA] [FE] feat(pairing): reword settings card and reorder Opik URL/Workspace * Add caption "The CLI tried to pair using these Opik settings:" above the card so the framing is unambiguous — these are the values the CLI used, not the user's current session. * Rename "Pairing with Opik at" → "Opik URL" so the labels match the CLI banner (Opik URL / Workspace) verbatim. * Reorder rows: URL first, then Workspace — same order as the banner. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * [NA] [SDK][FE] feat(pairing): include project name in pair link and context card * CLI build_pairing_link now accepts project_name and appends a URL-encoded `&project=<name>` query param. run_pairing forwards it through. * PairingPage reads `project` from the query; PairingStatusScreen renders a new Project row under Workspace in the error context card. * Project names with spaces / `/` round-trip safely via percent-encoding; added a regression test covering the encoding shape. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * fix lint * fix(pairing): scheme allow-list on Opik URL link + URL-encode workspace Addresses PR #6691 review feedback: * PairingStatusScreen: validate `expectedBaseUrl` with `URL()` and only render it as a clickable `<a>` when the scheme is http/https; otherwise render as plain text. Prevents a crafted pair URL like `?url=javascript:alert(1)` from producing a clickable script link. * PairingPage: thread expected workspace/project/baseUrl through the `invalid_link` branch as well, so the CLI context card surfaces even when the link fragment is malformed. * PairingStatusScreen: loosen `showWorkspaceContext` to fire when any of workspace/project/url is present; gate the Workspace row individually. * pairing.py: URL-encode the `workspace` query param the same way as `url` and `project`, so workspace names containing `&`/`=` don't split the query and confuse the FE's URLSearchParams parsing. Added a regression test covering the encoding. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * fix(tui): use solid bullet for banner mark The combining-enclosing-circle glyph (⠀⃝) failed to render on terminals that lack the combining mark, falling back to two red placeholder boxes before "opik". Switch to U+25CF (●), which renders consistently as a solid colored dot. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * refactor(preflight): skip redundant project lookup when preflight already 404'd should_create_project now returns (create_if_missing, known_missing) so the interactive path — which already observed the 404 before prompting — can tell resolve_project_id to skip the duplicate lookup and go straight to _create_project. Headless still leaves known_missing=False so the resolver can find an already-existing project across re-runs. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * [OPIK-6497] [SDK][FE] feat(runner): notify backend on close for instant disconnect * Supervisor calls disconnect_runner on shutdown (Ctrl+C, SIGTERM, 410 eviction, clean exit) so the FE flips the runner card off before the heartbeat TTL expires. Best-effort: server-side cleanup is idempotent and the heartbeat-TTL reaper picks up anything we miss. * activate.py restores default signal handlers and raises KeyboardInterrupt on first signal so a second Ctrl+C force-exits when the child wedges on its own SIGINT path. Bound supervisor's graceful timeout to 5s. * Tighten FE poll interval to 1s so the disconnect surfaces in the runner card immediately. * Cover supervisor disconnect on shutdown, 410 eviction, ApiError tolerance, and end-to-end SIGINT/SIGTERM delivery via a subprocess driver. Implements OPIK-6497. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * revert(activate): drop SIG_DFL + KeyboardInterrupt force-kill path The two-stage signal handling (raise KeyboardInterrupt on first signal, restore SIG_DFL so a second signal force-kills) was a safety net for wedged agents that swallow KeyboardInterrupt. Removing it: the supervisor's graceful timeout + SIGKILL escalation already covers wedged children, and keeping the handler purely cooperative avoids changing the SIGINT contract that frameworks (uvicorn, etc.) may rely on. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * [OPIK-6516] [SDK] feat(runner): add `opik connect/endpoint stop` to cleanly terminate local runners Adds first-class stop subcommands so Ollie (and humans) can tear down a headless runner without resorting to pkill on the process group, which often left orphaned children and stale "Endpoint open" state in the Agent Sandbox. - `opik connect stop --project X | --all | --runner ID` and the same for endpoint. Each Supervisor writes a small JSON lock file under `~/.opik/runners/` on startup (pid, runner_id, runner_type, project, workspace, started_at) and removes it in the shutdown finally; stale entries are purged opportunistically on the next write. - Stop sends SIGTERM to the supervisor pid, which routes through the OPIK-6497 handler that calls `disconnect_runner` before exit — the FE flips off on its next poll instead of waiting out the heartbeat TTL. SIGKILL is reserved for unresponsive supervisors and skips the backend notification (the reaper picks them up). - `opik connect` and `opik endpoint` become Click groups with a hidden `_run` subcommand and a shared `RunnerGroup.resolve_command` fallback, so the legacy `opik endpoint --project X -- python script.py` invocation form still works alongside the new `stop` subcommand. `--project` is now required at the Click layer instead of validated by hand. - Headless endpoint pairing now surfaces a "Paired ✓" panel with a direct Agent Playground link (connect keeps the generic project URL since Ollie is the user-facing surface there). Both browser and headless flows share `pairing.post_pairing_url` so the destination logic lives in one place. - Internal helpers (Supervisor, launch_supervisor) drop default values on parameters that aren't part of the user SDK API — every caller passes workspace/project_name/runner_type explicitly. - Intra-package imports in `cli/local_runner/` switched to module-style (`from . import pairing; pairing.RunnerType`) for clearer call-site attribution and easier patch targets in tests. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * ci(guardrails): disable guardrails-ai integration tests guardrails-ai is on PyPI quarantine, so the install step in the lib integration runner fails before pytest even starts. Comment out the job and its references in workflow_dispatch and notify-slack `needs`; drop the SUITE_RESULTS entry (a YAML `#` would survive as literal text into the JSON payload). Re-enable when the package is back on PyPI. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * style(runner): collapse RunnerGroup fallback call onto one line ruff format wants the super().resolve_command(...) call on a single line; restore that to unblock CI lint. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * fix(runner): address PR review on opik <type> stop CLI - stop.py: only drop the pid lock file when the runner actually exited. On SIGTERM-denied / SIGKILL-failed / still-alive paths we leave the file in place so a subsequent `opik <type> stop` can rediscover the live supervisor instead of being blind to it. - pid_file.remove(): bump the OSError log line to LOGGER.error — a failed pid-file unlink is a real signal that the cleanup contract broke, not a debug-level event. - RunnerGroup: narrow the `UsageError`-catch fallback. Connect (no positional after `_run`) now surfaces click's native "No such command 'stp'" for subcommand typos instead of routing them into `_run` and printing a confusing "Missing --project". Endpoint opts in via `accepts_positional_after_run=True` so the legacy `opik endpoint -- python script.py` form keeps surfacing the helpful "Missing --project" error when `--project` is omitted. - connect.py / endpoint.py: split `--runner` help text across lines and reword it as a usage hint ("Use when a project has more than one runner attached to it") instead of the opaque "disambiguates". Brings source lines back under the 88-char project limit. - test_pid_file.py and test_stop.py: rename every test method to the python-sdk testing convention (`test_<WHAT>__<CASE>__<EXPECTED_RESULT>` / `__happyflow`). Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]> * [NA] [BE] fix: stop BaseRedisSubscriberTest.shouldRemoveConsumerOnStop flaking (#6754) * [NA] [BE] fix: stop BaseRedisSubscriberTest.shouldRemoveConsumerOnStop flaking BaseRedisSubscriber.stop() calls removeConsumer().block(longPollingDuration) to wait for Redis to remove the consumer from the group, then disposes consumerScheduler. If the block timeout fires first, the in-flight removeConsumer call is cancelled by the disposal and the consumer is left behind. TestStreamConfiguration had longPollingDuration = 100ms, which is tighter than Redis-container response under load (CI or otherwise), so the test failed deterministically locally (8/8) and intermittently in CI. - Bump test longPollingDuration to 2s (prod default is 5s; this keeps tests fast for the happy path while matching prod's headroom ratio). - Handle the now-reachable empty-group case: stream.listConsumers(...) returns an empty Mono (block() -> null) when the group has no consumers, so wrap with blockOptional().orElse(List.of()). Stress-verified locally: 10/10 passing after fix (was 0/8 before). * fix(tests): use 500ms longPollingDuration to keep both LifecycleTests and RetryTests green The previous 2s bump fixed shouldRemoveConsumerOnStop but extended the read-loop cycle past the 2s test budget, so autoClaim never fired and shouldAckAndRemoveAfterMaxRetries / shouldHandleMixedSuccessRetryableAnd- NonRetryableMessagesInSameBatch broke instead. longPollingDuration is the XREADGROUP BLOCK timeout AND stop()'s removeConsumer block timeout, so it has to be: - long enough that Redis-container response under load doesn't exceed it (or stop() cancels the in-flight removal on scheduler disposal), and - short enough that one chain finishes inside the 2s test wait so autoClaim (gated by claimIntervalRatio polls) actually runs. 500ms gives ~5x headroom for Redis response and keeps the cycle short enough: with claimIntervalRatio=2 (the retry-tests override), autoClaim fires around t ~= 1.2s, well inside 2s. Stress-verified locally: 10/10 BUILD SUCCESS running both LifecycleTests and RetryTests nested classes. * [OPIK-6519] [BE] perf: replace FINAL with LIMIT 1 BY in trace threads closing candidate query (#6748) * [OPIK-6519] [BE] perf: replace FINAL with LIMIT 1 BY in trace threads closing candidate query The TraceThreadsClosingJob runs FIND_PENDING_CLOSURE_THREADS_SQL every 15s, and on large self-hosted deployments (Uber: 132M rows, 36+ parts on trace_threads) the FROM trace_threads FINAL + GROUP BY exceeds the 30s reactor timeout. Under high-volume ingestion the reactor cancels but the underlying ClickHouse query keeps running, R2DBC buffers accumulate, and the JVM crashes with OutOfMemoryError. Replaces the FINAL+GROUP BY dedup with the same LIMIT 1 BY reverse-order pattern introduced in OPIK-4828 for the thread-view query. Measured on opik_prod (16.37M rows, 10 parts), median wall time drops from ~2.9s to ~0.28s (~10x), data read from 2.68 GiB to 76 MiB (~36x), memory from 130 MiB to 36 MiB (~4x), with the identical result set verified on the same now() snapshot and on 997 threads with active/inactive transitions in the last 24h (100% agreement on latest status between FINAL and LIMIT 1 BY). * chore(threads): add log_comment to pending-closure query Routes FIND_PENDING_CLOSURE_THREADS_SQL through getSTWithLogComment so the ClickHouse log_comment placeholder is populated, matching the rest of the DAO. Addresses baz-reviewer feedback on PR #6748. * [NA] [BE] Apply Spotless formatting to backend sources (#6750) * [NA] [BE] Apply Spotless formatting to backend sources Reformat 53 Java files under apps/opik-backend to match Spotless rules (Google Java Format). Pure whitespace changes around @JsonView annotation arrays and record/class braces; no behavioral changes. * [NA] [INFRA] Add Spotless sweep SHA to .git-blame-ignore-revs Append the branch-head SHA for the backend Spotless reformat (6fcfa6f8a8) so git blame skips it. Note: under squash-merge, the SHA that actually lands on main will differ from the branch head, so a follow-up commit on main may still be needed to record the merged SHA. * add hiding demo project selector * add mask to Back button * add additional masks * added additional step for projects test * [NA] [CI] [GHA] security: use npm ci instead of npm install in Docker and lint workflows (#6755) * Use npm ci in opik-frontend Dockerfile (post TanStack incident) * Use npm ci in frontend_linter workflow (post TanStack incident) * Use npm ci in typescript_sdk_linter and publish_cursor_extension workflows (post TanStack incident) Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> --------- Co-authored-by: Daniel Dimenshtein <[email protected]> Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]> * Update versions to 2.0.40 and bump base version to 2.0.41 * [OPIK-6581] [BE] [CI] fix: quote GITHUB_OUTPUT and GITHUB_STEP_SUMMARY redirects (#6749) * [OPIK-6456] [BE] feat: add prompt masks for non-destructive prompt resolution (#6731) * [OPIK-6456] [BE] feat: add prompt masks for non-destructive prompt resolution Add version_type ENUM('prompt_version','mask') discriminator column on prompt_versions, defaulting existing rows to 'prompt_version'. This is the schema foundation for mask overlays: masks live in the same table as regular prompt versions but are filtered out of latest_version and version_count queries, and selectable via mask_id at resolution time. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * feat(prompts): add PromptVersionType enum and versionType field Introduces the PROMPT_VERSION/MASK discriminator at the API layer. versionType defaults to PROMPT_VERSION so existing clients are unaffected; mask creation will set it to MASK on the embedded PromptVersion in CreatePromptVersion. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * feat(prompts): filter masks from list/count queries and support mask_id resolution in findById - prompt_versions.version_type='prompt_version' filters added to: - latest_version and version_count subqueries in PromptDAO findById/find/findByIds/count - PromptDAO.findByCommit and findPromptsByCommits (commits are prompt-version-only) - PromptVersionDAO.find/findCount when listing by prompt_id (id-based lookups still return any type so masks remain reachable by id) - PromptVersionDAO.findByCommit - PromptDAO.findById gains an optional mask_id that, when present, populates requested_version with that version row (service validates it is actually a mask) - INSERT in PromptVersionDAO.save writes the new version_type column - PromptVersionColumnMapper reads version_type from JSON_OBJECT payloads with a PROMPT_VERSION fallback for older serialized rows Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * feat(prompts): mask-aware getById and bulk retrieveVersionsByIds in service - getById(UUID, UUID maskId): delegates to PromptDAO.findById with the optional mask_id. When a mask_id is supplied and no row matches for the given prompt, throws NotFoundException so callers see a clean 404 instead of a Prompt with a null requestedVersion. - retrieveVersionsByIds(List<UUID>): bulk version lookup powering the upcoming POST /v1/private/prompts/retrieve endpoint. Returns versions in whatever order PromptVersionDAO produces (id DESC); missing ids are dropped from the response. - Existing getById(UUID) delegates to the new overload with maskId=null, so all current callers keep their behavior. - Variable enrichment factored into enrichWithVariables() and applied to both latestVersion and requestedVersion on the resolution path. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * feat(prompts): mask_id query param on GET /prompts/{id} and POST /prompts/retrieve - GET /v1/private/prompts/{id} accepts an optional mask_id query param. When set, the response Prompt has requestedVersion populated with the matching mask version (404 if no such version belongs to this prompt). - POST /v1/private/prompts/retrieve accepts { ids: [...] } and returns List<PromptVersion>. Powers the FE flow that resolves multiple mask overlays in a single round-trip. - PromptVersionIdsRequest validates 1..1000 non-null UUIDs. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * feat(prompts): suppress PROMPT_COMMITTED alerts and lastUpdatedAt bump on mask saves Masks are non-destructive overlays, not real commits. They should not trigger PROMPT_COMMITTED alert subscribers and should not move the parent prompts last_updated_at, which the UI surfaces as recent prompt activity. Both side effects are now gated on versionType != MASK; regular prompt_version saves are unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * test(prompts): cover mask creation, isolation, mask_id resolution, bulk retrieve New PromptMasks nested class in PromptResourceTest: - mask creation via POST /prompts/versions with version_type=mask - masks excluded from latestVersion and versionCount on GET /prompts/{id} - masks excluded from GET /prompts/{id}/versions and from commit-based lookups (/by-commit and /retrieve-by-commits) - GET /prompts/{id}?mask_id=... populates requestedVersion with the mask while keeping latestVersion pointed at the latest real version - 404 when mask_id is unknown or belongs to a different prompt - POST /prompts/retrieve returns versions by id and rejects empty bodies - creating a mask does not bump the parent prompts lastUpdatedAt Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * remove unused import * feat(local-runners): carry multiple prompt masks per job via prompt_masks Extends CreateLocalRunnerJobRequest and LocalRunnerJob with promptMasks, a Map<promptId, maskId> that lets a local-runner job overlay masks on several prompts in a single run. The existing single maskId field stays in place but is marked @Deprecated for both DTOs. EndpointJobServiceImpl persists prompt_masks as JSON on the Redis job hash and restores it via parsePromptMasks on every job read path (nextJob, getJob, listJobs). LocalRunnersResourceTest covers create + claim round-trips for the new field and that promptMasks is null when not provided. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * bump migration version * address comments * bump migration number * [NA] [SDK] [DOCS] Update automatically OpenAPI spec and Fern code (#6766) * [NA] [SDK] fix: preserve environment when merging opik_args (#6735) * Create cleanup_e2e_docker.yaml * [OPIK-6543] Fix MEMORY_LIMIT_EXCEEDED in span/trace usage queries (#6760) * [OPIK-6499] [QA] feat: add SDK clients and use public Opik TS SDK for backend inspection (#6768) * [OPIK-6499] [QA] feat: add SDK clients and use public Opik TS SDK for backend inspection Add the opik npm package as a runtime dependency of the e2e suite. The package will back both the new sdkClient.typescript half (OPIK-6499) and the cutover of core/backend/client.ts away from openapi-fetch (the OPIK-6126 roll-in deferred to this ticket). openapi-fetch stays in devDependencies for now because core/backend/client.ts still imports it; it is removed in a follow-up commit after that file is rewritten. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * feat(sdk-clients): add PythonSdkClient and PythonSdkBridgeError * feat(sdk-clients): add makeTypescriptSdk factory * feat(sdk-clients): add SdkClient interface and makeSdkClient factory * feat(backend-client): swap openapi-fetch for new Opik().api.projects.* * chore(deps): remove openapi-fetch (replaced by opik SDK) * test(sdk-clients): seed test for cross-client wiring and error mapping --------- Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]> * [NA] [DOCS] Fix broken documentation links (#6769) * [NA] [DOCS] fix: resolve broken links across docs and redirects - Fix /tracing/ollie → /ollie (observability, agent_sandbox, self_improving_agents) - Fix /tracing/supported_models → /tracing/advanced/cost_tracking (integrations, concepts) - Fix /tracing/opentelemetry/overview → /integrations/opentelemetry (new integrations) - Fix agent_optimization/* → development/optimization-runs/* paths (latest version restructure) - Fix /reference/python-sdk/overview → external SDK reference URL (log_traces) - Fix TypeScript SDK evaluation page cross-links (evaluate/evaluatePrompt slugs) - Fix /prompt_engineering/getting-started → /development/agent-configuration/* paths - Fix changelog links (opik-university slug, log_traces path) - Add missing metric pages (g_eval_conversation_metrics, structure_output_compliance) to docs-v2 - Update docs.yml redirect destinations from agent_optimization/* to development/optimization-runs/* - Update docs.yml tracing/cost_tracking → tracing/advanced/cost_tracking redirect destination Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * [NA] [DOCS] fix: remove {#id} anchors from MDX heading causing acorn parse error MDX in the latest Fern version treats {#...} in headings as JSX expressions. Acorn rejects #id as invalid JS, breaking the preview build. Strip the anchors; Fern auto-generates slug anchors from the heading text anyway. Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * [NA] [DOCS] ci: retrigger docs preview build Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * [NA] [DOCS] fix: remove invalid frontmatter key causing acorn parse error pytest\_codeblocks\_skip with literal backslashes is not a valid JS identifier. Fern exports frontmatter as a JS object, so acorn fails on the backslash. This field is a pytest-codeblocks testing directive; Fern ignores it. Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * [NA] [DOCS] fix: same acorn fixes applied to v1 docs/ counterparts - Remove {#id} anchors from docs/evaluation/metrics/g_eval_conversation_metrics.mdx - Remove pytest\_codeblocks\_skip frontmatter from docs/evaluation/metrics/structure_output_compliance.mdx Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * [NA] [CI] align fern-api version to 4.71.5 across all environments fern.config.json and the deploy workflow both pin 4.71.5, but the preview workflow used 0.64.26, the OpenAPI auto-update workflow was unpinned (latest), and package.json resolved to 0.64.26 via ^0.94.6. Pinning everything to 4.71.5 so local dev, preview CI, and deploy CI all use the same version. Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * [NA] [DOCS] Remove Gretel integration and fix broken links in preview - Delete Gretel integration pages (docs/ and docs-v2/) and remove from nav — Gretel no longer exists - Fix playground.mdx: /v1/configuration/ai_providers → /v1/workspace-settings/ai_providers (section was renamed) - Fix langserve.mdx: /v1/tracing/integrations/langchain → /v1/integrations/langchain (integrations moved to own tab) Co-Authored-By: Claude Sonnet 4.6 <[email protected]> --------- Co-authored-by: Claude Sonnet 4.6 <[email protected]> * [OPIK-6532] [SDK] feat: cascade optimizations on opik migrate dataset (Slice 5) (#6746) * [OPIK-6532] [SDK] feat: cascade optimizations on opik migrate dataset Stops dropping `optimization_id` during the migrate cascade. Recreates every optimization referencing the source dataset under the destination project with a fresh UUID, populates `plan.optimization_id_remap`, and re-points each experiment's `optimization_id` FK at the new destination optimization id so the destination's Optimization Studio shows the same rows and trial groupings as the source. - New `CascadeOptimizations` planner action ordered between `ReplayVersions` and `CascadeExperiments` - New `cascade_optimizations` module: paginated `find_optimizations` enumeration, per-row recreate with fidelity fields (name, objective_name, status, metadata, studio_config, last_updated_at), READ-ONLY aggregates (num_trials, baseline_*, best_*, etc.) omitted - Experiment cascade re-points `optimization_id` via the remap; orphan path increments `experiments_with_orphan_optimization_id` counter - `recreate_experiment` forwards `optimization_id` on the migrate path - Per-optimization `migrate_optimization` audit records with source_id -> destination_id mapping; uses `optimization_status` key so the audit-level status field isn't shadowed - Unit tests: planner ordering, optimization fidelity, RO-aggregate guard, empty optimization, pagination, 409 propagation, audit shape, studio-config Read->Write reconstruction, experiment FK re-point, orphan path, no-source-id path - E2E test: 1 optimization + 2 trial experiments + 1 control experiment round-trip with grouping preserved at destination Implements OPIK-6532: opik migrate dataset (Slice 5): cascade optimizations * fix(migrate): render CascadeOptimizations in plan table + progress bar Two UX gaps spotted running the cascade against staging: - `_print_plan` had no branch for `CascadeOptimizations`, so the dry-run plan table skipped row 4 entirely (jumped from 3 to 5) even though the action executed correctly. Add the missing branch. - `_cascade_optimizations` ran silently without a Rich progress bar, jarring next to the verbose experiment-cascade bar that fires immediately after. Thread a `progress_callback` through `cascade_optimizations` and drive a Rich Progress block in the executor that mirrors `_cascade_experiments`'s shape (one tick per optimization). Single-level bar -- no nested per-optimization detail bar -- because production datasets carry tens to low-hundreds of optimizations at most. The algorithmic core stays console-agnostic; the bar is driven by the per-optimization callback so unit tests don't need Rich in the loop. Verified end-to-end on staging: plan table now lists `cascade optimizations -> project <target>` as row 4, and a "Cascading optimizations" Rich bar fires for the optimization phase between "Replaying versions" and "Cascading experiments". * fix(migrate): zero-opt progress bar + share ProgressCallback type alias Two follow-ups from PR review (#6746): - Fix zero-optimization progress-bar bug: the empty-cascade path fired one terminal `progress_callback(0, 0, "done")`, which created the Rich task at `total=1` but never advanced `completed`, leaving the bar at 0% for an already-finished migration. Set `completed = max(total, 1)` whenever the "done" tick fires (including first-callback creation) so the bar renders at 100% on both the empty and populated paths. - Hoist `ProgressCallback` / `InnerProgressCallback` type aliases into a new `datasets/_progress.py` shared module. Removes the previous drift surface where the same alias was redefined in `experiments.py` + `optimizations.py` and re-spelled inline as `Callable[[int, int, str], None]` in `version_replay.py`. Single source of truth + cross-module convention documented (esp. `label == "done"` finalization). Re-exported from `experiments.py` with `# noqa: F401` so any external imports keep working. * [OPIK-6455] [BE] feat: add environment ownership to prompt versions (#6767) * Add environment to prompts * fix * fix(prompt-envs): require existing env on PATCH and address review nits Address PR #6767 review comments: - PATCH /versions/{id} now returns 404 when the environment does not exist in the workspace, instead of silently auto-creating (which could no-op at the workspace env cap and leave a version pointing at an unregistered env). Adds EnvironmentService.existsByName + DAO countByName. - getPromptById's environment query param now validates @Pattern + @Size, matching PromptVersionRetrieve / PromptVersion. - Migration 000072: add adjacent comment above CREATE UNIQUE INDEX and a trailing newline, per migrations.md. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * test(prompt-envs): route new tests through PromptResourceClient Replace direct client.target(...) calls in the new PromptEnvironments nested block with PromptResourceClient helpers (matching the convention used elsewhere in PromptResourceTest): - callGetPrompt overload now accepts environment - callCreatePromptVersion / callGetPromptVersion / getPromptVersion - callSetPromptVersionEnvironment - callRestorePromptVersion Removed local getVersionById helper; postVersionRaw and patchVersionEnvironment now delegate to the client. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * refactor(prompt-envs): address review nits on PATCH env endpoint - Rename PATCH /versions/{versionId} -> /versions/{versionId}/environments to scope the endpoint to environment ownership. - Move the mask+environment cross-field constraint to a class-level @AssertTrue on PromptVersion so it is enforced at request validation; remove the redundant service-side check on POST. - Drop LOWER() from EnvironmentDAO.countByName: the environments table uses utf8mb4 (case-insensitive default collation) and has a UNIQUE (workspace_id, name) index, so the LOWER() wrap was both unnecessary and index-defeating. - Extract withEnvironmentPromotionLock helper shared by createVersionWithEnvironment and setVersionEnvironment. - Update tests for the new path and the 422 status for the AssertTrue violation. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * style: format PromptVersion @AssertTrue annotation * [NA] [SDK] [DOCS] Update automatically OpenAPI spec and Fern code (#6772) * [OPIK-6500] [QA] feat: add base fixtures for Opik 2.0 E2E suite (#6770) * [OPIK-6500] [QA] feat: add base fixtures for Opik 2.0 E2E suite * feat(fixtures): add base fixture with worker-scoped clients and testNamespace * feat(fixtures): add project fixture with SDK-create REST-delete * feat(fixtures): add scratch-dir fixture with per-test isolation * feat(fixtures): add failure-artifacts collector with capture API * feat(fixtures): add composition barrel as canonical import path * test(fixtures): add local smoke test for staging verification * test(fixtures): remove local smoke after staging verification * fix(artifacts): preserve only on failed/timedOut, not skipped/interrupted * feat(playwright): wire FastAPI bridge via webServer directive Playwright now spawns the opik-sdk-driver bridge automatically at the start of every test run, polls /health until ready, and shuts it down on exit. Subsequent test tickets (OPIK-6128 onwards) no longer need to manually start the bridge for staging verification — npx playwright test just works. reuseExistingServer is true outside CI so a locally-running bridge (from 'uv run uvicorn ...' in another terminal during development) is reused instead of spawned twice. env block spreads process.env then layers OPIK_URL_OVERRIDE and OPIK_WORKSPACE on top. The bridge's Python SDK convention is OPIK_URL_OVERRIDE; the e2e suite uses OPIK_BASE_URL — the mapping happens here so test authors set one URL var. Verified against staging: full seed suite passes with bridge auto-spawned (3/3 sdk-clients tests, 6.6s, no manual uvicorn start). * [OPIK-6524][BE] Cost Tracking: Add Missing Models and Providers - Elastic Inference Service OTEL (#6759) * [OPIK-6524][BE] Add support for Elastic Inference Service model/provider resolution in OpenTelemetry pipeline and cost calculations - Introduced `ElasticInferenceServiceResolver` to rewrite EIS provider/model attributes to underlying Opik canonical values for accurate cost lookup and downstream filtering. - Added `model_prices_overrides.json` to define pricing and alias mappings for EIS-supported models. - Updated `OpenTelemetryMapper` to integrate EIS resolution logic, ensuring original provider/model values are recorded in span metadata for traceability. - Included comprehensive tests for EIS integration, alias resolution, and dot-to-hyphen normalization in cost calculations. - Introduced end-to-end tests ensuring correct attribute rewriting, provider/model mapping, and non-zero cost verification for supported EIS models. * [OPIK-6524][BE] Fixed formatting issue in ElasticInferenceServiceResolver * [OPIK-6524][BE] Update log level and formatting for model price overrides error handling in `CostService` * [OPIK-6524][BE] Refactor alias handling in `CostService` to ensure order-independent overrides - Introduced two-pass processing logic to separate direct overrides and alias resolution. - Utilized `List<Map.Entry<String, ModelCostData>>` for intermediate storage of alias mappings to ensure deterministic behavior during iteration. * [OPIK-6524][BE] Consolidate EIS provider/model mapping tests into parameterized test - Replaced individual test cases with a single `@ParameterizedTest` using `CsvSource` for provider/model mapping validation. - Simplified test logic to ensure consistency across multiple provider prefixes. * [OPIK-6524][BE] Updated code comments to clarify the logic behind direct overrides and alias resolution ordering. * Update versions to 2.0.41 and bump base version to 2.0.42 * [OPIK-6311] [BE] perf: slim count path + bypass traces hop in target-projects pre-query (#6756) * [OPIK-6311] [BE] perf: slim count path + bypass traces hop in target-projects pre-query Two complementary fixes for the experiment-comparison page-load latency: 1. DatasetItemVersionDAO.SELECT_DATASET_ITEM_VERSIONS_WITH_EXPERIMENT_ITEMS_COUNT — add a <if(slim_count)> branch that reads count(DISTINCT stable_dataset_item_id) directly from experiment_item_aggregates, pre-pruning the dataset_item_versions FINAL lookup by (workspace_id, dataset_id). The gate fires when push_top_limit would also fire AND there are no filters/search — i.e. the data side already takes the fast path, but the count was forced through the heavy CTE chain. 2. ExperimentDAO.SELECT_TARGET_PROJECTS — read project_id directly from the experiments table when populated, fall back to experiment_aggregates.project_id for experiments without it on the row, and only traverse experiment_items -> traces for the experiments missing from BOTH (legacy fallback). Removes the workspace-wide experiment_items + traces scan that was the dominant cost. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * [OPIK-6311] [BE] fix: scope slim count to requested dataset via experiment_aggregates Address review feedback (Baz HIGH on DatasetItemVersionDAO.java:461): the <if(slim_count)> branch was only filtering experiment_item_aggregates by workspace_id, so EIA rows from other datasets in the same workspace would be counted. EIA has no dataset_id column, so add an IN subquery against experiment_aggregates (whose sorting key starts with (workspace_id, dataset_id, id) → no FINAL needed). The optional experiment_ids predicate now lives inside that subquery and stays consistent with the legacy branch's scoping. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * [OPIK-6311] [BE] chore: drop SQL line-comments from SELECT_TARGET_PROJECTS The CTE names already convey the intent (eia_projects, legacy_scope, legacy_trace_scope, legacy_projects) and the surrounding Javadoc on the constant explains the fast/fallback layering. The inline comments were redundant. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * [OPIK-6311] [BE] perf: extend slim_count to non-search filtered counts Drops the no-filter precondition on the slim_count gate. Mirroring top_dataset_items in the data path, the slim count now renders: - EXPERIMENT_ITEM filters directly on eia - FEEDBACK_SCORES_AGGREGATED + ..._IS_EMPTY filters on eia.feedback_scores - DATASET_ITEM filters via a dataset_items_filtered_ids CTE matched against eia.dataset_item_id IN arrayJoin([id, row_id]) (same stable-id resolution shape the data path uses) Gate is now `hasAggregated && !hasRaw && !hasSearch`. Search still falls back to the legacy heavy path because it requires the trace-side join. Benchmarked on a production-shaped fixture (workspace with ~2.3M items in EIA): no filter 3.4 s, EXPERIMENT_ITEM filter 0.4 s, FEEDBACK_SCORES_AGGREGATED 0.5 s, AGGREGATED_IS_EMPTY 1.2 s, combined EIA filters 0.6 s. DATASET_ITEM filter still pays the DIV pre-resolve scan but skips the heavy upstream CTEs. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * [OPIK-6311] [BE] fix: skip ZERO_UUID aggregate project_id in SELECT_TARGET_PROJECTS When populateAggregations runs before any traces exist for an experiment, experiment_aggregates.project_id is stored as ZERO_UUID (00000000-0000-0000-0000-000000000000). The legacy SELECT_TARGET_PROJECTS computed projects from the traces table, so it correctly returned an empty set in that state. The earlier rewrite read experiments.project_id directly and returned the experiment's project_id even with no traces, which then narrowed downstream trace filters to a project_id that had no trace data — causing FIND_GROUPS / FIND_GROUPS_AGGREGATIONS to return empty (ExperimentAggregatesIntegrationTest.experimentsWithZeroUuidAggregateProjectIdAreVisibleViaFallback). The fix: - Drop the experiments.project_id UNION branch entirely. - Filter ZERO_UUID out of eia_projects via AND project_id != :zero_uuid. - Fall back to the experiment_items -> traces traversal for any experiment without a valid (non-ZERO_UUID) aggregate. This matches the legacy "traces are the source of truth" semantics: the target_project_ids set only includes projects backed by actual trace data. On observed prod workspaces this drops the result from a buggy 10 (including ZERO_UUID and three projects with no traces) to the correct 6, latency ~1.3 s vs the legacy ~23 s. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * [OPIK-6311] [BE] fix: dedup eia_projects via sorting key instead of FINAL experiment_aggregates is ReplicatedReplacingMergeTree on (workspace_id, dataset_id, id) with last_updated_at as the version column. The earlier eia_projects CTE projected non-key columns (project_id) without deduplication, so it could read a stale project_id from a prior version row and incorrectly suppress the experiment_items -> traces fallback via legacy_scope. Switch to the same LIMIT 1 BY sorting-key dedup pattern OPIK-6519 uses for trace_threads: read all candidate rows, sort by sorting key DESC + last_updated_at DESC, LIMIT 1 BY the sorting key, then filter ZERO_UUID on the deduped result. Same latency as FINAL on this query (the IN clause keeps the candidate set small) without the merge cost. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * [OPIK-6311] [BE] style: bind zero_uuid as UUID, matching other call sites Address review nit: every other zero_uuid bind in this file passes the UUID object directly (lines 1940, 1971, 2003, 2209, 2251, 2619); the new getTargetProjectIdsForExperiments site was the only one passing .toString(). Functionally equivalent, just consistent. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * [OPIK-6311] [BE] docs: note slim_count CTE divergence on the constant javadoc Address review nit: flag that the slim branch's dataset_items_filtered_ids mirrors the push_top_limit one minus the dataset_version_id predicate. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]> * [OPIK-6574] [SDK] feat: add opik migrate prompt command (Slice 6) (#6752) * [OPIK-6574] [SDK] feat: add opik migrate prompt command (Slice 6) Introduce ``opik migrate prompt NAME --to-project=B`` as a peer to ``opik migrate dataset``. Moves a single prompt (with its full version history) from one project to another within the same workspace. The plan emits RenameSource then CreateDestination then ReplayVersions: - Rename frees the workspace-unique name; re-passes description + tags since the BE rename PUT wipes description (no COALESCE in PromptDAO). - CreateDestination omits ``template`` so the BE does NOT auto-mint a v1 (PromptService.create skips version creation when template is empty), letting the replay loop carry every source commit verbatim. - ReplayVersions paginates ``get_prompt_versions``, reverses to oldest-first (BE orders pv.id DESC; UUIDv7 ids -> chronological), and POSTs each version with the source ``commit`` carried verbatim. Fresh dest prompt_id means the (workspace_id, prompt_id, commit) unique key never collides. Shared helpers (``_create_destination_prompt`` and ``_replay_prompt_versions``) are exposed module-level so Slice 7 (OPIK-6575, dataset cascade pulling prompts) can import them un…

[NA] [SDK] fix: preserve environment when merging opik_args

4cd2d73

github-actions Bot added python Pull requests that update Python code tests Including test files, or tests related like configuration. Python SDK labels May 17, 2026

pragnyanramtha marked this pull request as ready for review May 17, 2026 03:39

pragnyanramtha requested a review from a team as a code owner May 17, 2026 03:39

Copilot AI review requested due to automatic review settings May 17, 2026 03:39

baz-reviewer Bot approved these changes May 17, 2026

View reviewed changes

Copilot AI reviewed May 17, 2026

petrotiurin requested changes May 18, 2026

View reviewed changes

pragnyanramtha mentioned this pull request May 18, 2026

opik_args span merge drops StartSpanParameters fields #6761

Closed

Merge remote-tracking branch 'upstream/main' into pragnyanramtha/NA-p…

eb9220b

…reserve-opik-args-environment

pragnyanramtha changed the title ~~[NA] [SDK] fix: preserve environment when merging opik_args~~ [NA] [SDK] fix: preserve StartSpanParameters when merging opik_args May 18, 2026

Merge branch 'main' into pragnyanramtha/NA-preserve-opik-args-environ…

7660951

…ment

petrotiurin approved these changes May 19, 2026

View reviewed changes

petrotiurin merged commit 6e22288 into comet-ml:main May 19, 2026
36 checks passed

natagh23 pushed a commit that referenced this pull request May 20, 2026

[NA] [SDK] fix: preserve environment when merging opik_args (#6735)

0fc16eb

jverre pushed a commit that referenced this pull request Jun 12, 2026

[NA] [SDK] fix: preserve environment when merging opik_args (#6735)

53ac400

Navigation Menu

Repository navigation

[NA] [SDK] fix: preserve StartSpanParameters when merging opik_args - #6735

[NA] [SDK] fix: preserve StartSpanParameters when merging opik_args#6735
petrotiurin merged 3 commits into
comet-ml:mainfrom
pragnyanramtha:pragnyanramtha/NA-preserve-opik-args-environment

pragnyanramtha commented May 17, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

petrotiurin left a comment

Uh oh!

pragnyanramtha commented May 18, 2026

Uh oh!

pragnyanramtha commented May 18, 2026

Uh oh!

petrotiurin left a comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

pragnyanramtha commented May 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Details

Change checklist

Issues

AI-WATERMARK

Testing

Documentation

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

petrotiurin left a comment

Choose a reason for hiding this comment

Uh oh!

pragnyanramtha commented May 18, 2026

Uh oh!

pragnyanramtha commented May 18, 2026

Uh oh!

petrotiurin left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

pragnyanramtha commented May 17, 2026 •

edited

Loading