Skip to content

test_summary: count pytest xfailed/xpassed so expected-fail runs are observed - #41

Merged
JARosen merged 1 commit into
thruwire:mainfrom
kevin9327:fix/pytest-xfail-summary
Oct 5, 2026
Merged

JARosen merged 1 commit into
thruwire:mainfrom
kevin9327:fix/pytest-xfail-summary

Conversation

@kevin9327

Copy link
Copy Markdown
Contributor

Pytest summary lines can name expected failures and unexpected passes:

2 xfailed in 0.12s
1 passed, 2 xfailed, 1 xpassed in 0.40s

parse_pytest_summary_line only counted passed / failed / error(s) / skipped. An xfailed/xpassed token was ignored, so a run whose only outcomes were expected failures returned None — and ObservationBuilder then recorded no test_results for that worker output. The factory looked like the worker never ran tests.

A mixed line undercounted too: 1 passed, 2 xfailed became passed=1, total=1 instead of three known outcomes.

Change: treat xfailed and xpassed like skipped (known non-failing outcomes on the summary line). Warnings and deselected counts stay ignored. Docstring updated.

Tests: two cases in test_parse_pytest_summary_line_exact_outcomes. Both fail on main and pass with this change.

python -m pytest -q tests/test_test_summary.py   # 36 passed
python -m ruff check src/foreman/test_summary.py tests/test_test_summary.py

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@JARosen
JARosen merged commit a725944 into thruwire:main Oct 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants