test(e2e): cover add-fallback flow in Router Settings#29069
Conversation
The Router Settings → Fallbacks → Add Fallbacks flow was an uncovered manual-QA path. This adds a test that opens the modal, picks a primary + fallback from the seeded mock models, saves, and verifies both render in the fallback table.
Greptile SummaryAdds an end-to-end Playwright test for the Router Settings → Fallbacks → Add Fallback flow, covering a previously manual-QA-only path. The test seeds
Confidence Score: 5/5New test-only file with no changes to production code; safe to merge. The change adds a single new Playwright spec with no impact on production code paths. The teardown logic correctly reads from the live router via /get/config/callbacks (which does return router_settings) and posts back only the filtered fallbacks. All previously flagged weaknesses — missing teardown, weak table assertion, mismatched tab-count comment — have been addressed in this revision. No files require special attention.
|
| Filename | Overview |
|---|---|
| ui/litellm-dashboard/e2e_tests/tests/settings/routerSettings.spec.ts | New e2e test covering the Add Fallback flow in Router Settings, with idempotent before/after teardown, race-condition-safe waitForResponse pattern, and strengthened dual-filter table assertion. |
Reviews (3): Last reviewed commit: "fix(e2e): keyboard-select fallback model..." | Re-trigger Greptile
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
…by text - Match `.ant-select-item-option` by text instead of `getByTitle(...)` — FallbackGroupConfig uses `options=` (not <Select.Option> children), so no `title` attribute is emitted and the title-based selector hangs. - Add before/after hooks that wipe any fallback for fake-openai-gpt-4 via /config/update so retries and local reruns don't trip on leftover state. - Tighten the success assertion to a single tbody row containing BOTH the primary and the fallback names — pre-existing rows can no longer vacuously satisfy the check. - Fix the stale "Three tabs" comment to "Four tabs". Addresses Greptile P2s on PR #29069.
|
@greptileai re review |
- Replace mouse-based option clicks with click-to-focus + type + Enter. FallbackGroupConfig's Selects use `options=` and a custom getPopupContainer, so locating options via `.ant-select-dropdown` hit several races: DOM-clicks left antd's popup state stale (the primary popup then intercepted the fallback click), `getByRole` matched always-mounted hidden options, and pointer stability fought the open animation. Typing into the showSearch input narrows the listbox to one option and Enter selects it cleanly. - Assert on dialog-side state changes (the active tab adopts the primary model name; the chain helper shows "1/10 used") instead of popup contents — these reflect the actual selection landing. - Cleanup helper now hits /get/config/callbacks (the real endpoint; /get/callbacks returns 404), so the before/after reset actually clears prior router_settings.fallbacks state.
|
@greptileai re review |
b0ea013
into
litellm_internal_staging
* test(e2e): cover add-fallback flow in Router Settings as proxy admin The Router Settings → Fallbacks → Add Fallbacks flow was an uncovered manual-QA path. This adds a test that opens the modal, picks a primary + fallback from the seeded mock models, saves, and verifies both render in the fallback table. * fix(e2e): make router-fallback test idempotent and pick antd options by text - Match `.ant-select-item-option` by text instead of `getByTitle(...)` — FallbackGroupConfig uses `options=` (not <Select.Option> children), so no `title` attribute is emitted and the title-based selector hangs. - Add before/after hooks that wipe any fallback for fake-openai-gpt-4 via /config/update so retries and local reruns don't trip on leftover state. - Tighten the success assertion to a single tbody row containing BOTH the primary and the fallback names — pre-existing rows can no longer vacuously satisfy the check. - Fix the stale "Three tabs" comment to "Four tabs". Addresses Greptile P2s on PR BerriAI#29069. * fix(e2e): keyboard-select fallback models + correct cleanup endpoint - Replace mouse-based option clicks with click-to-focus + type + Enter. FallbackGroupConfig's Selects use `options=` and a custom getPopupContainer, so locating options via `.ant-select-dropdown` hit several races: DOM-clicks left antd's popup state stale (the primary popup then intercepted the fallback click), `getByRole` matched always-mounted hidden options, and pointer stability fought the open animation. Typing into the showSearch input narrows the listbox to one option and Enter selects it cleanly. - Assert on dialog-side state changes (the active tab adopts the primary model name; the chain helper shows "1/10 used") instead of popup contents — these reflect the actual selection landing. - Cleanup helper now hits /get/config/callbacks (the real endpoint; /get/callbacks returns 404), so the before/after reset actually clears prior router_settings.fallbacks state.
Summary
Adds an e2e test for the previously-uncovered manual-QA item "Router Settings → Fallbacks → Add fallback". The test opens the Add Fallbacks modal, picks
fake-openai-gpt-4as primary andfake-anthropic-claudeas the fallback (both seeded in the e2e config), saves, and verifies both render in the fallback table.Test plan
e2e_tests/tests/settings/routerSettings.spec.ts::Add a fallback and verify it appears in the tableRefs LIT-3093