Repository navigation
fix(agent-sre): prefer higher quality at equal pareto cost - #4211
Conversation
Signed-off-by: Naveen Chatlapalli <[email protected]>
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
github-actions
Bot
added
size/S
|
🟡 Contributor Check: MEDIUM
Automated check by AGT Contributor Check. |
github-actions
Bot
added
the
needs-review:MEDIUM
PR Review Summary
Verdict: AI review comments are untrusted advisory output. The summary reports workflow-generated completion status only, not model-authored pass/fail claims. |
|
Verified in code and locally: a cheap-first registration kept both points on main; the Before I approve, two process items that apply to all nine PRs in this batch: please use the repository template with its Checklist, Attribution, AI Assistance and IP sections, and tick the attestations in your own words. The AI note says Codex prepared and submitted the changes at your request; CONTRIBUTING asks that no PR be submitted by an agent without a human reviewing the specific changes, so one sentence confirming you reviewed this diff before it was opened settles it. Once that and the gated checks are green I will approve and merge. |
|
I reviewed the specific changes in this diff before the PR was opened. I have updated the description to use the repository template sections, completed the attestations with the validation scope and limitations, and clarified the AI-assistance note. |
MohammadHaroonAbuomar
left a comment
MohammadHaroonAbuomar
left a comment
There was a problem hiding this comment.
Approving at c8bfb4c. Pareto selection prefers higher quality at equal cost; test reproduces the tie-break bug on main and passes here; ruff clean. Description uses the repository template with the attestations completed, and the author confirmed reviewing the diff before opening the PR. All 12 pull_request workflow runs on this head completed successfully. Commits are signed off.
…Optimizer.recommend (#4469) recommend sorted feasible models by cost alone, so among models with the same cost the one registered first was marked is_optimal and chosen by suggest_routing, even when another had a higher quality score and was the only one of the two on the Pareto frontier. Break ties by quality, as #4211 did for pareto_frontier. Signed-off-by: Shivam Sharma <[email protected]> Co-authored-by: Claude Opus 5.5 <[email protected]>
Description
When equal-cost models arrive in lower-quality-first order, the Pareto frontier retains the lower-quality model as well as the better one. Reversing input order drops the dominated model, making the result depend on registration order.
Break equal-cost sorting ties by descending quality before scanning the frontier, and clarify the dominance description.
Validation
The lower-quality-first regression fails before the fix; the reverse ordering is a control. Optimizer tests: 19 passed. Full package run: 1,496 passed, 3 skipped, 1 known baseline failure (the separate Windows benchmark clock bug).
All regressions were reproduced on origin/main
406ddd4dbefore the production change. Whitespace checks pass. Ruff passes on the changed files. Changed-line spelling passes with the existing API identifierDeduplicatorallowed in the temporary check input.Scope: the existing Agent-SRE source tree, which remains exercised by repository CI. Combined validation of these six independent SRE fixes: 1,505 passed, 3 skipped on Windows/Python 3.12.
Type of Change
Package(s) Affected
Checklist
Attribution & Prior Art
Prior art / related projects: Uses the existing Agent-SRE implementation and standard library operations. No code was copied or adapted from an external project.
AI Assistance
I reviewed the specific changes in this diff before this PR was opened.
Codex assisted with implementation, regression tests, and local validation under my direction.
IP, Patents, and Licensing
Related Issues
Newly reproduced defect; the reproduction and regression coverage are described above.