Describe the bug

CI is failing due to running out of disk space https://github.com/apache/arrow-datafusion/actions/runs/3995952868/jobs/6855600796

This correlates with enabling the dyn_arith_dict feature in CI - #4999

To Reproduce
Steps to reproduce the behavior:

Expected behavior

We should not exceed the disk capacity of the runners, either by splitting up the CI into smaller steps and cleaning the target directory inbetween, or splitting the CI into multiple smaller workflows like we do for arrow-rs

Additional context
Add any other context about the problem here.