Stricter Clippy checks in CI #18467

Is your feature request related to a problem or challenge?

Now datafusion CI is using the default set of lint rules in Clippy, see https://github.com/apache/datafusion/blob/main/ci/scripts/rust_clippy.sh

Here is the Clippy doc for lint categories and levels https://doc.rust-lang.org/nightly/clippy/

There are several categories not enabled by default, and according to the Clippy doc, the reasons are they're 'annoying' for general codebases, or they have occasional false positives.

I think DataFusion is both performance and correctness-critical, and it also includes substantial complexity. Stricter lint checks could help improve code maintainability. We could hand-pick some useful optional lint rules and include them in the CI.

Clippy Lints list: https://rust-lang.github.io/rust-clippy/master/index.html

Here are some example lints that are not enabled now, but I think are useful:
https://rust-lang.github.io/rust-clippy/master/index.html#needless_pass_by_value
https://rust-lang.github.io/rust-clippy/master/index.html#todo
https://rust-lang.github.io/rust-clippy/master/index.html#missing_errors_doc

Tasks

• Figure out a list of suitable Clippy rules to enforce in datafusion
• [Tracking] Rollout of new lint clippy::needless_pass_by_value in all datafusion crates #18503

Describe the solution you'd like

No response

Describe alternatives you've considered

No response

Additional context

No response