Skip to content

ci: add slt-only test coverage - #26219

Merged
alamb merged 2 commits into
apache:mainfrom
2010YOUY01:slt-coverage
Oct 29, 2026
Merged

alamb merged 2 commits into
apache:mainfrom
2010YOUY01:slt-coverage

Conversation

@2010YOUY01

@2010YOUY01 2010YOUY01 commented Oct 12, 2026 •

Copy link
Copy Markdown
Member

Which issue does this PR close?

  • Closes #.

Rationale for this change

Add test coverage report for individual sqllogictest runs. For motivation, see PR diff file docs/source/contributor-guide/testing.md

Demo

See the bot reply in the current PR,

image

It means Datafusion now reaches 66% test line coverage from slt alone -- you can click into codecov website to further browse each file's coverage from slt alone, by selecting the flag.

What changes are included in this PR?

  1. Add a command to run slt tests, and save coverage report locally
  2. In CI, add a job to run such command, and upload it, and codecov bot will reply with report.

What is the testing strategy for this PR?

Verify if the codecov bot reply in this PR works

Are there any user-facing changes?

no

@2010YOUY01
2010YOUY01 marked this pull request as draft October 12, 2026 08:00
@github-actions github-actions Bot added documentation Improvements or additions to documentation development-process Related to development process of DataFusion labels Oct 12, 2026
@codecov-commenter

codecov-commenter commented Oct 12, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.80%. Comparing base (de06656) to head (37380f7).
⚠️ Report is 277 commits behind head on main.

Flag Coverage Δ
all-tests 82.79% <100.00%> (?)
sqllogictests 66.73% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

@@            Coverage Diff             @@
##             main   #26219      +/-   ##
==========================================
- Coverage   82.80%   82.80%   -0.01%     
==========================================
  Files        1149     1149              
  Lines      452234   454165    +1931     
  Branches   452234   454165    +1931     
==========================================
+ Hits       374484   376049    +1565     
- Misses      54952    55264     +312     
- Partials    22798    22852      +54     
Files with missing lines Coverage Δ
xtask/src/ci_steps.rs 86.15% <100.00%> (+0.37%) ⬆️

... and 155 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@2010YOUY01
2010YOUY01 marked this pull request as ready for review October 14, 2026 09:36

@kosiew kosiew left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@2010YOUY01,

Thanks for adding separate SQLLogicTest coverage reporting. This addresses the need to identify gaps in end-to-end SQL test coverage that aggregate coverage can conceal. The implementation is focused, and the additional complexity is justified by the independent measurement it provides.

I see no blocking concerns. Approving.

@alamb alamb left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @2010YOUY01 and @kosiew

Comment thread .github/codecov.yml

# Upload code coverage report, GitHub bot will comment with the code coverage report.
#
# It has two reports:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we move all the code coverage jobs to this workflow (rather than some in rust?)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue is that now the default test run and coverage generation are combined to save CI time.

Maybe it's better to move this job to rust.yml, and combine it with default coverage job to save some compile time 🤔 will give it a try later.

# Conflicts:
#	xtask/src/ci_steps.rs
@alamb
alamb enabled auto-merge October 29, 2026 21:10
@alamb
alamb added this pull request to the merge queue Oct 29, 2026
Merged via the queue into apache:main with commit f8983af Oct 29, 2026
45 checks passed
@2010YOUY01

Copy link
Copy Markdown
Member Author

Thank you for the review @kosiew and @alamb

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

development-process Related to development process of DataFusion documentation Improvements or additions to documentation v56.0.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants