fix(tier3): keep skill lift consistent when rebuilding reports - #212

Preserve activation eligibility alongside judge-aware lift, retain unavailable CLI comparisons, and keep both changelog entries. Add regression coverage for activated and never-activated saved reports.

Signed-off-by: Christopher Kevin <[email protected]>

Merge main 3728e2f (#215 custom execution backends, #212 judge-aware
lift, v0.6.0 release).

Conflicts:
- src/skillevaluator/tier3/harbor/runner.py: the unsupported env_mode
  failure keeps the PR's coded INVALID_CONFIGURATION result with main's
  _supported_env_modes() list (built-in plus installed custom backends).
  The runtime-plan failure keeps the PR's coded result, followed by
  main's new try/except around the agent_import_paths selection.
- CHANGELOG.md (auto-merged): git placed the PR's error-code entry in
  the released 0.6.0 section; moved it to Unreleased under ### Added.
  Main's 0.6.0 section is unchanged.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Narendran Raghavan <[email protected]>

Brings in main up to 3728e2f: #212 judge-aware lift, 15adf98 (schema
"unavailable" rejection) and #215 custom execution backends.

Conflicts:
- docs/cli-reference.mdx: kept the branch's experimental hermes agent note
  and main's custom execution backends sentence on --env-mode.
- src/skillevaluator/cli.py: import MAX_TRIAL_RETRIES and
  PLUGIN_LOAD_CHOICES with main's all_env_modes; --env-mode lists
  installed backends.
- src/skillevaluator/tier3_environments.py: kept the branch's plugin-load
  and trial-retry constants and main's custom backend registry.
- src/skillevaluator/tier3/harbor/collector.py: main's strategy-aware
  aggregate matcher keeps the branch's N/A-metric filter; main's
  constituent judge models merge runs before the branch's
  artifact-aware failure and N/A merges.
- src/skillevaluator/tier3/harbor/runner.py: an explicit agent import path
  may be a native plugin wrapper or a custom backend's wrapper; agent
  import paths and the arm base paths are resolved inside main's backend
  error handling before native plugin wrappers are chosen. The arm base
  for a custom backend is the backend's own agent wrapper (else the stock
  agent), as build_harbor_run_command launches it, so a native wrapper
  over an unknown base falls back or refuses.
- src/skillevaluator/evaluation/tier3_report.py: both helper sets kept.
  Lift data uses main's type check and the branch's activation-aware
  intervals; dimensions take main's judge_lift and the branch's baseline
  N/A flag; evaluators keep lift None without a comparison (main's
  judge exclusion included). When judge-mixed cases exist, the
  effectiveness headline (or the legacy members comparison) and its basis
  come from the matching-judge comparison (#212), the case-weighted
  interval that pairs those cases is dropped, partial and never-activated
  comparisons still get no lift, and the lift note says when no matching-
  judge comparison remains. Main's _dimension_score calls use the
  branch's weighted_dimension_score.
- src/skillevaluator/reporting/cli.py: the branch's None-safe evaluator
  table renders a missing score, baseline or lift as main's "n/a".
- tests/reporting/test_reporters.py: both test sets kept.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Narendran Raghavan <[email protected]>

Main's #212 renders an unavailable baseline or lift in the CLI evaluator
table as "n/a" (its test_render_agent_eval_with_unavailable_comparison
checks this). The branch's test for a run without a baseline still
expected "N/A" in that table; it now expects main's wording. The
Markdown table keeps "N/A".

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Narendran Raghavan <[email protected]>

Merge main 3728e2f (#215 custom execution backends, #212 judge-aware
lift, v0.6.0 release).

Conflicts:
- src/skillevaluator/tier3/harbor/runner.py: the unsupported env_mode
  failure keeps the PR's coded INVALID_CONFIGURATION result with main's
  _supported_env_modes() list (built-in plus installed custom backends).
  The runtime-plan failure keeps the PR's coded result, followed by
  main's new try/except around the agent_import_paths selection.
- CHANGELOG.md (auto-merged): git placed the PR's error-code entry in
  the released 0.6.0 section; moved it to Unreleased under ### Added.
  Main's 0.6.0 section is unchanged.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Narendran Raghavan <[email protected]>

Brings in main up to 3728e2f: #212 judge-aware lift, 15adf98 (schema
"unavailable" rejection) and #215 custom execution backends.

Conflicts:
- docs/cli-reference.mdx: kept the branch's experimental hermes agent note
  and main's custom execution backends sentence on --env-mode.
- src/skillevaluator/cli.py: import MAX_TRIAL_RETRIES and
  PLUGIN_LOAD_CHOICES with main's all_env_modes; --env-mode lists
  installed backends.
- src/skillevaluator/tier3_environments.py: kept the branch's plugin-load
  and trial-retry constants and main's custom backend registry.
- src/skillevaluator/tier3/harbor/collector.py: main's strategy-aware
  aggregate matcher keeps the branch's N/A-metric filter; main's
  constituent judge models merge runs before the branch's
  artifact-aware failure and N/A merges.
- src/skillevaluator/tier3/harbor/runner.py: an explicit agent import path
  may be a native plugin wrapper or a custom backend's wrapper; agent
  import paths and the arm base paths are resolved inside main's backend
  error handling before native plugin wrappers are chosen. The arm base
  for a custom backend is the backend's own agent wrapper (else the stock
  agent), as build_harbor_run_command launches it, so a native wrapper
  over an unknown base falls back or refuses.
- src/skillevaluator/evaluation/tier3_report.py: both helper sets kept.
  Lift data uses main's type check and the branch's activation-aware
  intervals; dimensions take main's judge_lift and the branch's baseline
  N/A flag; evaluators keep lift None without a comparison (main's
  judge exclusion included). When judge-mixed cases exist, the
  effectiveness headline (or the legacy members comparison) and its basis
  come from the matching-judge comparison (#212), the case-weighted
  interval that pairs those cases is dropped, partial and never-activated
  comparisons still get no lift, and the lift note says when no matching-
  judge comparison remains. Main's _dimension_score calls use the
  branch's weighted_dimension_score.
- src/skillevaluator/reporting/cli.py: the branch's None-safe evaluator
  table renders a missing score, baseline or lift as main's "n/a".
- tests/reporting/test_reporters.py: both test sets kept.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Narendran Raghavan <[email protected]>

Main's #212 renders an unavailable baseline or lift in the CLI evaluator
table as "n/a" (its test_render_agent_eval_with_unavailable_comparison
checks this). The branch's test for a run without a baseline still
expected "N/A" in that table; it now expects main's wording. The
Markdown table keeps "N/A".

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Narendran Raghavan <[email protected]>