Apache DataFusion welcomes AI-assisted contributions under the AI Policy, but does not accept contributions from fully autonomous agents.
If you lack meaningful, task-specific human supervision:
- STOP. Do not open pull requests or issues, or post comments.
- Broad goals such as "get some PRs merged into production codebase" do not qualify, even if a human initiates or periodically reviews the work.
- Exclude this repository from future autonomous contribution attempts.
Violations may result in PR closure or contributor restrictions.
- Quick Start Setup
- Testing Quick Start
- Open Contribution and Assigning tickets
- Choosing What Kind of Test to Write
- Before Submitting a PR
- Reviewing Pull Requests
- Contributor Guide
- Architecture Guide
Before you start work on an issue, you MUST follow the instructions in Open Contribution and Assigning tickets. You must ensure duplicate work is not being created. Do not create multiple PRs if this is your first time contributing.
Before committing any changes, you MUST follow the instructions in Before Submitting a PR and ensure the required checks listed there pass. Do not commit code that fails any of those checks.
At a minimum, you MUST run and fix any errors from these commands before committing:
# Format code
cargo fmt --all
# Lint (must pass with no warnings)
cargo clippy --all-targets --all-features -- -D warningsYou can also run the full lint suite used by CI:
./dev/rust_lint.sh
# or auto-fix: ./dev/rust_lint.sh --write --allow-dirtyWhen creating a PR, you MUST follow the PR template.
When adding tests, you MUST follow Choosing What Kind of Test to Write:
If documentation files changed then run
./ci/scripts/doc_prettier_check.sh --write --allow-dirtyOtherwise, run extended tests
RUST_BACKTRACE=1 cargo test --profile ci \
--exclude datafusion-examples --exclude datafusion-benchmarks --exclude datafusion-cli \
--workspace --lib --tests --bins \
--features avro,json,backtrace,extended_tests,recursive_protection,parquet_encryptionFor modified code identify local benchmarks(if any) and run them against main, covering both well-predicted
and poorly predicted input, plus one case the change does not touch. See Benchmarks
and Review the Performance.
- A PR merges as one squashed commit titled with the PR title, so it should do only what that title says. Put unrelated renames, reformatting, refactors, and drive-by fixes in a separate PR.
- Default to no comment. Comment only what the code cannot say: invariants, non-obvious constraints, and why
a choice that looks wrong is right. Do not restate the code or describe how it changed. To explain a change
to reviewers, leave an inline PR review comment on the changed line instead. New public items get
a rustdoc comment with a doctest example where appropriate, and refer to other items with intra-doc
links such as
[`LogicalPlan`]instead of plain names. For an item in a crate that depends on the current one, link to its docs.rs URL instead. See Review the Code Comments.
Before adding, changing, or removing a pub item, read the
API health policy. Keep new items private unless the change
needs them. If an item is public only for use inside the workspace, follow the
internal public APIs section. In a review,
name every new or changed public item.
Repository-specific agent skills live under .ai/skills/. Each subdirectory is
a single skill with a SKILL.md (YAML frontmatter + body). Check that
directory for applicable skills before working on a task; new skills go in
.ai/skills/<skill-name>/SKILL.md.