Skip to content

Commit 72cbfd2

Browse files
committed
dev: add local CodeQL analysis for GitHub Actions
Add `ci/scripts/check_codeql_actions.sh`, which runs the default CodeQL query suite for GitHub Actions over the repository and writes a SARIF report. These are the same queries the "Analyze Actions" job in `.github/workflows/codeql.yml` runs. Register the script in `dev/rust_lint.sh` and document it in the testing guide. The workflow itself does not change. Part of apache#21048.
1 parent 0576a0b commit 72cbfd2

3 files changed

Lines changed: 259 additions & 0 deletions

File tree

‎ci/scripts/check_codeql_actions.sh‎

Lines changed: 233 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,233 @@
1+
#!/usr/bin/env bash
2+
#
3+
# Licensed to the Apache Software Foundation (ASF) under one
4+
# or more contributor license agreements. See the NOTICE file
5+
# distributed with this work for additional information
6+
# regarding copyright ownership. The ASF licenses this file
7+
# to you under the Apache License, Version 2.0 (the
8+
# "License"); you may not use this file except in compliance
9+
# with the License. You may obtain a copy of the License at
10+
#
11+
# http://www.apache.org/licenses/LICENSE-2.0
12+
#
13+
# Unless required by applicable law or agreed to in writing,
14+
# software distributed under the License is distributed on an
15+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
16+
# KIND, either express or implied. See the License for the
17+
# specific language governing permissions and limitations
18+
# under the License.
19+
20+
# Runs the default CodeQL query suite for GitHub Actions and writes a SARIF
21+
# report, the same queries the "Analyze Actions" job in the "CodeQL" workflow
22+
# runs. Nothing is uploaded. A zero exit status means the analysis completed,
23+
# not that the repository has no findings.
24+
25+
set -euo pipefail
26+
27+
SCRIPT_NAME="$(basename "${BASH_SOURCE[0]}")"
28+
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
29+
ROOT_DIR="$(cd "${SCRIPT_DIR}/../.." && pwd)"
30+
31+
# The pack default suite, which the hosted analysis also uses.
32+
QUERY_SUITE="codeql/actions-queries:codeql-suites/actions-code-scanning.qls"
33+
# Matches the hosted workflow's `category`.
34+
SARIF_CATEGORY="/language:actions"
35+
REPORT_PARENT="${ROOT_DIR}/target/codeql-actions"
36+
SETUP_DOCS="https://docs.github.com/en/code-security/how-tos/find-and-fix-code-vulnerabilities/scan-from-the-command-line/set-up-codeql-cli"
37+
38+
usage() {
39+
cat >&2 <<USAGE
40+
Usage: $0
41+
42+
Runs the default CodeQL GitHub Actions security suite over this repository and
43+
writes a SARIF report under target/codeql-actions/.
44+
45+
Needs the complete CodeQL bundle on PATH. The CLI-only download does not ship
46+
the query packs. See docs/source/contributor-guide/testing.md.
47+
48+
There is no --write mode, because a security finding needs a reviewed source
49+
change.
50+
-h, --help Show this message.
51+
USAGE
52+
exit 1
53+
}
54+
55+
# Reject unsupported arguments before any database or report directory exists.
56+
if [[ $# -gt 0 ]]; then
57+
case "$1" in
58+
-h|--help)
59+
usage
60+
;;
61+
*)
62+
echo "[${SCRIPT_NAME}] unsupported argument: $1" >&2
63+
usage
64+
;;
65+
esac
66+
fi
67+
68+
if ! command -v codeql &> /dev/null; then
69+
echo "[${SCRIPT_NAME}] codeql was not found on PATH. Install the complete CodeQL bundle, which ships the Actions extractor and the query packs, and add its codeql directory to PATH: ${SETUP_DOCS}" >&2
70+
exit 1
71+
fi
72+
73+
if ! CODEQL_VERSION="$(codeql version --format=terse 2>/dev/null)"; then
74+
echo "[${SCRIPT_NAME}] \`codeql version\` failed. Check the CodeQL installation on PATH: $(command -v codeql)" >&2
75+
exit 1
76+
fi
77+
78+
if ! command -v python3 &> /dev/null; then
79+
echo "[${SCRIPT_NAME}] python3 was not found on PATH. It reads the generated SARIF summary. Run the suite through uv, which provides Python: uv run ./dev/rust_lint.sh" >&2
80+
exit 1
81+
fi
82+
83+
if ! CODEQL_LANGUAGES="$(codeql resolve languages 2>/dev/null)"; then
84+
echo "[${SCRIPT_NAME}] \`codeql resolve languages\` failed. Check the CodeQL installation on PATH: $(command -v codeql)" >&2
85+
exit 1
86+
fi
87+
88+
# `grep` runs without `-q` so it reads all of its input, which keeps `pipefail`
89+
# from reporting a SIGPIPE-killed `codeql` as a failure on a match.
90+
if ! printf '%s\n' "${CODEQL_LANGUAGES}" | grep -E '^actions[[:space:]]' > /dev/null; then
91+
echo "[${SCRIPT_NAME}] this CodeQL installation has no \`actions\` extractor. Install the complete bundle rather than the CLI-only download: ${SETUP_DOCS}" >&2
92+
exit 1
93+
fi
94+
95+
# Proves the pack and its default suite resolve before a database is created.
96+
if ! codeql resolve queries "${QUERY_SUITE}" > /dev/null 2>&1; then
97+
echo "[${SCRIPT_NAME}] this CodeQL installation does not resolve ${QUERY_SUITE}. Install the complete bundle, which ships the pack: ${SETUP_DOCS}" >&2
98+
exit 1
99+
fi
100+
101+
ACTIONS_PACK_VERSION="$(codeql resolve qlpacks --format=json 2>/dev/null | python3 -c '
102+
import json
103+
import os
104+
import sys
105+
106+
try:
107+
packs = json.load(sys.stdin)
108+
paths = packs["codeql/actions-queries"]
109+
print(os.path.basename(paths[0].rstrip("/")))
110+
except Exception:
111+
print("unknown")
112+
')"
113+
114+
echo "[${SCRIPT_NAME}] CodeQL CLI ${CODEQL_VERSION}"
115+
echo "[${SCRIPT_NAME}] query pack codeql/actions-queries ${ACTIONS_PACK_VERSION}"
116+
echo "[${SCRIPT_NAME}] query suite ${QUERY_SUITE}"
117+
118+
# One report directory per run, so concurrent and repeated runs stay separate.
119+
mkdir -p "${REPORT_PARENT}"
120+
REPORT_DIR="$(mktemp -d "${REPORT_PARENT}/run-XXXXXX")"
121+
REPORT="${REPORT_DIR}/actions.sarif"
122+
CREATE_LOG="${REPORT_DIR}/database-create.log"
123+
ANALYZE_LOG="${REPORT_DIR}/database-analyze.log"
124+
125+
# Only this run's scratch directory is cleaned; the report directory is kept.
126+
SCRATCH_DIR=""
127+
cleanup() {
128+
if [[ -n "${SCRATCH_DIR}" ]]; then
129+
rm -rf "${SCRATCH_DIR}"
130+
fi
131+
}
132+
trap cleanup EXIT
133+
trap 'exit 130' INT
134+
trap 'exit 143' TERM
135+
SCRATCH_DIR="$(mktemp -d "${TMPDIR:-/tmp}/codeql-actions-db.XXXXXX")"
136+
DATABASE_DIR="${SCRATCH_DIR}/database"
137+
138+
echo "[${SCRIPT_NAME}] extracting the Actions database from ${ROOT_DIR}"
139+
if ! codeql database create "${DATABASE_DIR}" \
140+
--language=actions \
141+
--source-root="${ROOT_DIR}" 2>&1 | tee "${CREATE_LOG}"; then
142+
echo "" >&2
143+
echo "❌ [${SCRIPT_NAME}] CodeQL extraction failed. Log: ${CREATE_LOG}" >&2
144+
exit 1
145+
fi
146+
147+
echo "[${SCRIPT_NAME}] analyzing with ${QUERY_SUITE}"
148+
# `--no-download` keeps the analysis on the installed pack, off the network.
149+
if ! codeql database analyze "${DATABASE_DIR}" "${QUERY_SUITE}" \
150+
--format=sarif-latest \
151+
--sarif-category="${SARIF_CATEGORY}" \
152+
--no-download \
153+
--output="${REPORT}" 2>&1 | tee "${ANALYZE_LOG}"; then
154+
echo "" >&2
155+
echo "❌ [${SCRIPT_NAME}] CodeQL analysis failed. Log: ${ANALYZE_LOG}" >&2
156+
exit 1
157+
fi
158+
159+
# Malformed output is an error, and a missing field is never read as a severity.
160+
SUMMARY_STATUS=0
161+
python3 - "${REPORT}" <<'PYTHON' || SUMMARY_STATUS=$?
162+
import collections
163+
import json
164+
import sys
165+
166+
report_path = sys.argv[1]
167+
168+
try:
169+
with open(report_path, encoding="utf-8") as handle:
170+
report = json.load(handle)
171+
except (OSError, ValueError) as error:
172+
print(f"unreadable SARIF report {report_path}: {error}", file=sys.stderr)
173+
sys.exit(1)
174+
175+
if not isinstance(report, dict):
176+
print(f"malformed SARIF report {report_path}: top level is not an object", file=sys.stderr)
177+
sys.exit(1)
178+
179+
runs = report.get("runs")
180+
if not isinstance(runs, list):
181+
print(f"malformed SARIF report {report_path}: \"runs\" is missing or not a list", file=sys.stderr)
182+
sys.exit(1)
183+
184+
findings = []
185+
for run_index, run in enumerate(runs):
186+
if not isinstance(run, dict):
187+
print(f"malformed SARIF report {report_path}: run {run_index} is not an object", file=sys.stderr)
188+
sys.exit(1)
189+
results = run.get("results", [])
190+
if not isinstance(results, list):
191+
print(f"malformed SARIF report {report_path}: run {run_index} has a non-list \"results\"", file=sys.stderr)
192+
sys.exit(1)
193+
for result_index, result in enumerate(results):
194+
if not isinstance(result, dict):
195+
print(f"malformed SARIF report {report_path}: run {run_index} result {result_index} is not an object", file=sys.stderr)
196+
sys.exit(1)
197+
rule = result.get("ruleId")
198+
if not isinstance(rule, str) or not rule:
199+
print(f"malformed SARIF report {report_path}: run {run_index} result {result_index} has no \"ruleId\"", file=sys.stderr)
200+
sys.exit(1)
201+
location = "location unavailable"
202+
try:
203+
physical = result["locations"][0]["physicalLocation"]
204+
uri = physical["artifactLocation"]["uri"]
205+
line = physical.get("region", {}).get("startLine")
206+
location = f"{uri}:{line}" if isinstance(line, int) else uri
207+
except (KeyError, IndexError, TypeError):
208+
pass
209+
findings.append((rule, location))
210+
211+
if not findings:
212+
print("✅ CodeQL analysis completed with no findings.")
213+
sys.exit(0)
214+
215+
print(f"⚠️ CodeQL analysis completed with {len(findings)} finding(s):")
216+
print("")
217+
for rule, count in sorted(collections.Counter(rule for rule, _ in findings).items()):
218+
print(f" {rule}: {count}")
219+
print("")
220+
for rule, location in findings:
221+
print(f" {location}\t{rule}")
222+
PYTHON
223+
224+
if [[ ${SUMMARY_STATUS} -ne 0 ]]; then
225+
echo "" >&2
226+
echo "❌ [${SCRIPT_NAME}] could not read the SARIF report. Log: ${ANALYZE_LOG}" >&2
227+
exit 1
228+
fi
229+
230+
echo ""
231+
echo "[${SCRIPT_NAME}] report: ${REPORT}"
232+
echo "[${SCRIPT_NAME}] a successful run means the analysis completed, not that the repository is free of findings."
233+
echo "[${SCRIPT_NAME}] the report and its logs stay on disk. Remove them with: rm -rf ${REPORT_DIR}"

‎dev/rust_lint.sh‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -139,6 +139,7 @@ declare -a READONLY_STEPS=(
139139
"ci/scripts/check_large_files.sh|false"
140140
"ci/scripts/markdown_link_check.sh|false"
141141
"ci/scripts/security_audit.sh|false"
142+
"ci/scripts/check_codeql_actions.sh|false"
142143
"ci/scripts/check_circular_dependencies.sh|false"
143144
"ci/scripts/check_unused_dependencies.sh|false"
144145
"ci/scripts/rust_docs.sh|false"

‎docs/source/contributor-guide/testing.md‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -320,6 +320,31 @@ the README:
320320
[`dev/update_function_docs.sh`]: https://github.com/apache/datafusion/blob/main/dev/update_function_docs.sh
321321
[`ci/scripts/check_generated_docs.sh`]: https://github.com/apache/datafusion/blob/main/ci/scripts/check_generated_docs.sh
322322

323+
## CodeQL Actions Check
324+
325+
`ci/scripts/check_codeql_actions.sh` runs the default CodeQL query suite for
326+
GitHub Actions and writes a [SARIF] report under `target/codeql-actions/`, with
327+
the same queries as the "Analyze Actions" job in [`codeql.yml`].
328+
`./dev/rust_lint.sh` runs it, so the suite now needs CodeQL and pays the
329+
extraction and analysis cost. To run the check on its own:
330+
331+
```shell
332+
./ci/scripts/check_codeql_actions.sh
333+
```
334+
335+
It needs the complete [CodeQL bundle] on `PATH`, not the CLI-only download,
336+
which has no query packs. The baseline is CodeQL CLI 2.27.0, the default of the
337+
`github/codeql-action` commit pinned in [`codeql.yml`].
338+
339+
A zero exit status means the analysis completed, not that the repository has no
340+
findings. Findings do not fail the check. Remove old reports with
341+
`rm -rf target/codeql-actions`. Local and hosted SARIF differ, because only the
342+
hosted run uploads results and adds fingerprints.
343+
344+
[sarif]: https://docs.github.com/en/code-security/reference/code-scanning/sarif-support-for-code-scanning
345+
[codeql bundle]: https://docs.github.com/en/code-security/how-tos/find-and-fix-code-vulnerabilities/scan-from-the-command-line/set-up-codeql-cli
346+
[`codeql.yml`]: https://github.com/apache/datafusion/blob/main/.github/workflows/codeql.yml
347+
323348
## Benchmarks
324349

325350
### Criterion Benchmarks

0 commit comments

Comments
 (0)