Repository navigation
750 lines (710 loc) · 41.3 KB
/
Copy pathpattern-integrity.yml
File metadata and controls
750 lines (710 loc) · 41.3 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
name: pattern-integrity
# Permanent gate added 2026-05-27 after AZ caught v0.2.51 shipping 21 duplicate
# pattern IDs to PyPI (GLS-AW-148..168 were already in patterns.py from v0.2.42,
# the v0.2.51 ship script wrote them in AGAIN). The dogfood-fix + dedup landed
# in v0.2.52 but only AFTER damage was visible to users. This workflow makes
# the bug class impossible to merge again — runs on every PR + push to main.
#
# Pairs with:
# - tests/test_pattern_integrity.py (the test that catches dup IDs)
# - tests/test_dogfood_bugs.py (the 4 May 27 first-run regressions)
# - publish.sh integrity gate before twine upload
# - .git/hooks/pre-push (local same-gate, --no-verify bypass available)
#
# This workflow is the GitHub-side enforcement layer (no --no-verify bypass).
# 2026-09-10 split (T9 Fable + T8 Opus review, ASTRA NO-GO round 1 applied):
# the full six-Python matrix below took 2 h 50 m PER PULL REQUEST (the "Full
# test suite" step alone: 167 min per job). A contributor fixing a README typo
# waited three hours for a check. Now, on every pull request, four jobs run:
#
# classify — unfiltered, every PR. scripts/ci_classify.py — taken from
# origin/main, NEVER the PR's own copy — reads the COMPLETE change
# list from git (renames contribute both names) and says
# full=true unless EVERY path is documentation. Any error → full.
# GitHub's own path filters are NOT used: they inspect a bounded
# window of changed files and would fail open on large PRs.
# fast — always. Python 3.12, the same native tools, pinned media deps,
# receipt, media preflight and the SAME five blocking gates as
# the matrix (byte-identical run: lines), then the suite minus the
# two v0.5.6 acceptance modules. Minutes, not hours.
# integrity — the full matrix, unchanged, ONLY when classify says full=true
# (and always on push to main, tags, nightly, manual dispatch).
# coverage — always runs last and is the ONE check merge automation reads:
# it fails unless classify and fast succeeded AND, when full was
# required, all six matrix jobs succeeded. A skipped matrix only
# passes when classify affirmatively said docs-only, so requiring
# `coverage` never deadlocks a docs PR and never lets code skip
# the matrix.
#
# The release ship stays bound to the main-branch run of this workflow:
# SHIP_MANUAL "WATCH CI TO GREEN" waits on it after the scanner merge.
# tests/test_ci_gate.py loads this file as YAML and asserts the structure above
# (no path filters, no disabling `if` on classify/fast/coverage, the exact
# fast-suite ignore list, the gate commands, the coverage rule) and unit-tests
# the classifier, including ASTRA's four negative controls from round 1.
# Superseded PR runs are cancelled; runs on main and on tags are NOT. The release
# ship waits on the main-branch run and `certify` must exist for the exact release
# sha, so cancelling those would turn a gate into an absence — the one failure mode
# this workflow was rebuilt to remove.
concurrency:
# PER SHA ON A PUSH, per REF on a pull request, and the difference is the
# whole point. GitHub keeps only ONE QUEUED run per concurrency group, so
# while a matrix is running, a second main merge EVICTS the first one's
# queued run -- it is cancelled with zero jobs ever started, which reads
# afterwards exactly like a run that was never triggered. Measured
# 2026-09-19: of the last ten main runs only TWO completed all six
# integrity legs, and #202's merge commit 442419b lost its run that way,
# taking a prediction check with it.
#
# A pull request must keep the OPPOSITE behaviour: a new push there SHOULD
# supersede the previous run, which is why the sha is not appended
# unconditionally. A PR's `github.ref` is already `refs/pull/N/merge` and
# is stable across pushes, so the literal keeps PR grouping byte-identical.
#
# The commit that added this paragraph exists to BE a second push: the
# pre-merge half of the proof is that a new push to a pull request still
# supersedes the previous run, and a behaviour can only be shown by
# causing it.
group: ${{ github.workflow }}-${{ github.ref }}-${{ github.event_name == 'pull_request' && 'pr' || github.sha }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
on:
push:
branches: [main]
tags: ["v*"]
pull_request:
branches: [main]
schedule:
- cron: "17 9 * * *" # 09:17 UTC = 02:17 PT daily, before the 03:30 PT local gauntlet
workflow_dispatch:
# Sep-13 2026. This workflow declared no permissions at all, so every job here ran with
# whatever the repository default grants. release.yml and scorecard.yml have both had this
# since they were written; this is the third file catching up, and it is why OpenSSF
# Scorecard scored Token-Permissions 0 while two of three workflows were already correct.
# Nothing in this file writes. It reads the tree, installs, and runs tests.
permissions: read-all
jobs:
classify:
runs-on: ubuntu-24.04
permissions:
contents: read
timeout-minutes: 10
outputs:
full: ${{ steps.decide.outputs.full }}
quick: ${{ steps.lane.outputs.quick }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 0
- name: Decide whether the full matrix is required (fail-closed)
id: decide
env:
EVENT: ${{ github.event_name }}
BASE: ${{ github.event.pull_request.base.sha }}
HEAD: ${{ github.event.pull_request.head.sha }}
run: |
if [ "$EVENT" != "pull_request" ]; then
echo "full=true # event=$EVENT: not a pull request, the matrix always runs"
echo "full=true" >> "$GITHUB_OUTPUT"
exit 0
fi
# Judge with the classifier from the protected branch, never with the
# copy inside the PR under judgement (T8 review, 2026-09-10): a PR
# that edits scripts/ci_classify.py must not be able to classify
# itself as documentation. actions/checkout with fetch-depth: 0 makes
# origin/main available. If main has no classifier yet, fail to FULL.
if ! git show origin/main:scripts/ci_classify.py > "$RUNNER_TEMP/ci_classify.py" 2>/dev/null; then
echo "full=true # origin/main has no scripts/ci_classify.py; defaulting to FULL"
echo "full=true" >> "$GITHUB_OUTPUT"
exit 0
fi
python "$RUNNER_TEMP/ci_classify.py" --base "$BASE" --head "$HEAD"
# Which legs of the CLI matrix a pull request must run itself. The human and
# sarif legs of every cell re-run what the json leg already runs through
# another renderer (tests/test_v056_matrix.py, marker full_matrix), so a PR
# that cannot have changed a renderer runs the json legs only. Every push to
# main, every tag, the schedule and a manual run get quick=false and run all
# of it, and a PR that touches the CLI, the sarif writer, the shared result
# shape, the matrix itself, the marker registration, the gate test or this workflow gets
# quick=false too. Fail closed: anything but a clean "no match" is quick=false.
- name: Decide whether the human and sarif CLI legs can wait for main (fail-closed)
id: lane
env:
EVENT: ${{ github.event_name }}
BASE: ${{ github.event.pull_request.base.sha }}
HEAD: ${{ github.event.pull_request.head.sha }}
run: |
quick=false
GUARD='^(sunglasses/(cli|__main__|sarif|result)\.py|tests/(test_v056_matrix|v056_matrix|test_ci_gate)\.py|pytest\.ini|\.github/)'
if [ "$EVENT" = "pull_request" ] && files=$(git diff --name-only "$BASE" "$HEAD"); then
rc=0
grep -Eq "$GUARD" <<<"$files" || rc=$?
if [ "$rc" = "1" ]; then quick=true; fi
fi
echo "quick=$quick"
echo "quick=$quick" >> "$GITHUB_OUTPUT"
fast:
runs-on: ubuntu-24.04
permissions:
contents: read
# A hang must fail, not wait. Set from the first hosted measurement, not
# from local arithmetic (ASTRA round 1); tighten once measured.
timeout-minutes: 150
env:
SUNGLASSES_REQUIRE_FAST: "1"
# Four parts that together are exactly the fast suite (tests/ci_shard.py,
# the same split the integrity legs use). A single fast job was the longest
# check on a pull request, so every merge waited on it after the integrity
# legs had finished. The parts are named `fast (1)` to `fast (4)`;
# `needs.fast.result` is success only when ALL FOUR succeed, so classify,
# coverage and certify read it unchanged. Every part runs the same gate
# steps, so no gate depends on which part it landed in.
strategy:
fail-fast: false
matrix:
shard: ["1", "2", "3", "4"]
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Set up Python 3.12
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"
# Native decoders/OCR. Without these the v0.5.6 media tests cannot run at all,
# and a decoder that is missing turns a real finding into silence.
- name: Install native media tools (Tesseract, zbar, ffmpeg)
# A stalled mirror used to hold a runner for the whole job timeout. Each command is
# capped, and a retry against the same stuck mirror never passes, so every attempt
# after the first moves to the main Ubuntu archive before it tries again. The runner
# picks its mirror from apt-mirrors.txt, so that list is what gets rewritten. Three
# stalled attempts of 200 s each plus overhead run past 10 minutes; 12 lets the
# third one finish before the step cap cuts in.
timeout-minutes: 12
run: |
mirror_lines() {
grep -hE '^(URIs:|deb )' /etc/apt/sources.list /etc/apt/sources.list.d/*.sources 2>/dev/null || true
cat /etc/apt/apt-mirrors.txt 2>/dev/null || true
}
APT_OPTS="-o Acquire::http::Timeout=30 -o Acquire::Retries=2"
for attempt in 1 2 3; do
if [ "$attempt" -gt 1 ]; then
# An install stopped by the timeout leaves dpkg half-configured; repair it so the retry can run.
sudo timeout 120 dpkg --configure -a >/dev/null 2>&1 || true
echo "apt mirror before:"; mirror_lines
printf 'http://archive.ubuntu.com/ubuntu/\tpriority:1\nhttp://security.ubuntu.com/ubuntu/\tpriority:2\n' \
| sudo tee /etc/apt/apt-mirrors.txt >/dev/null
sudo sed -i 's|azure\.archive\.ubuntu\.com|archive.ubuntu.com|g' /etc/apt/sources.list /etc/apt/sources.list.d/*.sources 2>/dev/null || true
echo "apt mirror after:"; mirror_lines
fi
if sudo timeout 200 apt-get $APT_OPTS update -qq \
&& sudo timeout 200 apt-get $APT_OPTS install -y -qq tesseract-ocr libzbar0 ffmpeg; then
exit 0
fi
echo "apt attempt $attempt did not finish; retrying" >&2
done
exit 1
- name: Install package + dev + media deps
run: |
# PINNED PER INTERPRETER, not unpinned. pip 26.1 onward declares
# Requires-Python >= 3.10, so `pip==26.2.1` has no distribution for the
# 3.9 leg and that leg failed on this exact line. 26.0.1 is the newest
# pip 3.9 can take, read off the resolver's own list in the failing
# log rather than guessed. Markers rather than a shell branch: pip
# evaluates them and ignores the one that does not apply, so both pins
# stay visible as pins.
python -m pip install --upgrade \
'pip==26.2.1; python_version >= "3.10"' \
'pip==26.0.1; python_version < "3.10"'
# pytest stays unpinned in this fallback. The matrix spans Python 3.9 to 3.14 and a
# single exact pin cannot satisfy both ends: CI currently resolves 8.4.2 on the
# old legs and 9.1.1 on the new ones. Pillow is pinned per leg two lines below.
pip install -e ".[dev,media]" || { pip install -e . && pip install pytest "pytesseract==0.3.13" "PyPDF2==3.0.1" "pyzbar==0.1.9" Pillow; }
# Pin the reviewer's resolved stack rather than trusting free resolution.
# Pillow 12.3.0 requires >=3.10, so 3.9 necessarily takes 11.3.0 — that lane
# therefore differs from the graded stack on the exact library J1 concerns.
# It stays FULLY BLOCKING anyway: no exception is pre-authorised, and any
# difference there gets investigated before it is classified.
pip install "Pillow==12.3.0"
pip install "pytesseract==0.3.13" "PyPDF2==3.0.1" "pyzbar==0.1.9"
# The fast lane runs with the optional keyword-search extension and the
# integrity matrix runs without it, so both paths are exercised on every
# code change. tests/test_extension_parity.py compares them directly and,
# with SUNGLASSES_REQUIRE_FAST set, fails rather than skips if the
# extension is missing.
pip install pyahocorasick
# A version receipt, printed per job. Interpreter-dependent resolution can change
# frame-reading behaviour, which is the J1 surface — so the versions that actually
# ran are recorded, not assumed.
- name: Resolved environment receipt
run: |
echo "python : $(python -V 2>&1)"
echo "pip : $(pip -V)"
echo "pytest : $(pytest --version 2>&1 | head -1)"
python - <<'PYEOF'
import importlib.metadata as md
for pkg in ("Pillow", "pytesseract", "PyPDF2", "pyzbar"):
try:
print(f"{pkg:12s}: {md.version(pkg)}")
except Exception as exc:
raise SystemExit(f"MISSING media dependency {pkg}: {exc}")
PYEOF
echo "tesseract: $(tesseract --version 2>&1 | head -1)"
echo "ffmpeg : $(ffmpeg -version 2>&1 | head -1)"
echo "ffprobe : $(ffprobe -version 2>&1 | head -1)"
# Importability is not operation. A decoder that imports but never decodes
# reports a real finding as noise, so the modules are imported AND the two
# committed QR fixtures are actually decoded, each required non-empty.
- name: Media dependency preflight (import + PATH + real QR decode)
run: |
python - <<'PYEOF'
import shutil, sys
import PIL, pytesseract, PyPDF2 # noqa: F401
import pyzbar.pyzbar as zb
from PIL import Image
for tool in ("tesseract", "ffmpeg", "ffprobe"):
if shutil.which(tool) is None:
raise SystemExit(f"NATIVE TOOL MISSING on PATH: {tool}")
for fixture in ("tests/fixtures/v056/qr-ordinary.png",
"tests/fixtures/v056/qr-injection.png"):
with Image.open(fixture) as img:
codes = zb.decode(img.convert("RGB"))
if not codes or not codes[0].data:
raise SystemExit(f"QR CONTROL FAILED: {fixture} decoded to nothing "
f"-- zbar is present but not operational")
print(f"QR OK {fixture}: {len(codes)} code(s), {len(codes[0].data)} bytes")
print("media preflight OK")
PYEOF
- name: Pattern integrity tests (no dupes / id / category / shape)
run: pytest tests/test_pattern_integrity.py -v
- name: Dogfood regression tests (May 27 incident)
run: pytest tests/test_dogfood_bugs.py -v
- name: False-positive regression gate (Jun 6 incident — clean text must not block)
run: pytest tests/test_false_positives.py -v
- name: Customer-zero real-world suite (attacks block, normal usage passes)
run: python test_customer_zero.py
# v0.5.6 round-8 acceptance, as a DEDICATED BLOCKING step ahead of the full
# suite. These are the cells that discriminate the J1/J2 repair: 10 of them
# fail on the previous candidate. Exactly 19 must pass and NOTHING may skip --
# a skip here would let this workflow report success while omitting the very
# evidence the independent GO rests on.
- name: v0.5.6 round-8 acceptance (J1/J2) — 19 passed, zero skips
run: |
pytest tests/test_v056_round8.py -q -p no:cacheprovider \
| tee /tmp/round8.log
CODE=${PIPESTATUS[0]}
grep -qE "^19 passed" /tmp/round8.log || {
echo "ROUND-8 GATE INVALID: expected exactly '19 passed'"; exit 1; }
if grep -qE "[0-9]+ (skipped|error|failed)" /tmp/round8.log; then
echo "ROUND-8 GATE INVALID: a skip/error/failure is present"; exit 1; fi
exit "$CODE"
# The boundary harness tripwire, as a blocking step in both lanes so the
# fast lane still mirrors the matrix step for step. The harness mirrors
# the records the proxy writes, and this file fails when the two drift
# apart. It went red when revoking became a durable record and nothing
# ran it for three days. Only the three tests that need no local review
# material run here. The rest of the boundary suite reads scenario and
# review files that live outside the repository and runs nightly on the
# machine that holds them. Exactly 3 must pass and nothing may skip.
- name: Boundary harness tripwire, 3 passed and zero skips
run: |
pytest -q -p no:cacheprovider \
"gauntlet/boundary/tests/test_upstream_closed_mirrors_the_product.py::test_the_product_records_two_of_the_four_and_not_the_other_two" \
"gauntlet/boundary/tests/test_upstream_closed_mirrors_the_product.py::test_the_upstreams_list_changed_is_recorded_as_the_product_records_it" \
"gauntlet/boundary/tests/test_upstream_closed_mirrors_the_product.py::test_no_revoke_record_without_the_upstreams_list_changed" \
| tee /tmp/boundary_tripwire.log
CODE=${PIPESTATUS[0]}
grep -qE "^3 passed" /tmp/boundary_tripwire.log || {
echo "TRIPWIRE GATE INVALID: expected exactly '3 passed'"; exit 1; }
if grep -qE "[0-9]+ (skipped|error|failed|deselected)" /tmp/boundary_tripwire.log; then
echo "TRIPWIRE GATE INVALID: a skip/error/failure is present"; exit 1; fi
exit "$CODE"
# The receipts library's own tests live inside the package
# (sunglasses/receipts/tests), which pytest.ini's testpaths does not name,
# so no bare `pytest` ever collected them. Five of the seven files stopped
# importing on 2026-10-21 and nothing noticed. Named here as a blocking step
# in both lanes so the fast lane still mirrors the matrix step for step.
# At least one test must pass and nothing may skip, error, fail or be
# deselected; a count is not pinned, so adding a test needs no edit here.
# tests/test_ci_gate.py fails if a test_*.py under sunglasses/ is outside
# every path the gate runs.
- name: Receipts library tests (sunglasses/receipts/tests), zero skips
run: |
pytest -q -p no:cacheprovider sunglasses/receipts/tests \
| tee /tmp/receipts_lib.log
CODE=${PIPESTATUS[0]}
grep -qE "^[0-9]+ passed" /tmp/receipts_lib.log || {
echo "RECEIPTS GATE INVALID: no passing test was reported"; exit 1; }
if grep -qE "[0-9]+ (skipped|error|failed|deselected)" /tmp/receipts_lib.log; then
echo "RECEIPTS GATE INVALID: a skip/error/failure is present"; exit 1; fi
exit "$CODE"
# The fast suite: everything except the two v0.5.6 acceptance modules
# (tests/test_v056_matrix.py — the media matrix, 398 cases — and
# tests/test_repair_v056.py — 114 cases spanning media, consent, CLI output
# and result semantics) and tests/proxy. Those run in the `integrity`
# matrix whenever `classify` says the change is not documentation-only,
# and on every push to main, on tags, nightly and on demand.
#
# tests/proxy joined the list when its review fixtures were vendored into
# the repository: before that the real-pipe controls could not find their
# inputs and skipped, and once they could they ran, spawning child
# processes and waiting out deadline tests measured in whole minutes. The
# fast job hit its 60-minute cap at 58 minutes. Those controls belong in
# integrity, which runs the whole tree on every interpreter; a lane whose
# job is a 30-minute signal is not the place for a test that waits 61
# seconds on purpose.
#
# --ignore keeps pytest.ini's testpaths in force (no path argument), so
# the root-scope customer-zero collection is unchanged.
# tests/test_ci_gate.py asserts this exact list.
#
# The suite is split four ways by CI_SHARD (tests/conftest.py keeps part i
# and reports the rest as deselected), so the last line of each part's log
# reads "<kept> passed, ... <deselected> deselected" and kept + deselected
# is the unsharded count, the same number in all four logs. The four kept
# counts add up to it. tests/test_ci_shard_invariant.py proves the same
# property on the real collection.
- name: Fast suite (all tests except the v0.5.6 acceptance modules and tests/proxy)
env:
CI_SHARD: ${{ matrix.shard }}/4
run: |
pytest -q -p no:cacheprovider \
--ignore=tests/test_v056_matrix.py \
--ignore=tests/test_repair_v056.py \
--ignore=tests/proxy \
--durations=15
integrity:
needs: classify
if: needs.classify.outputs.full == 'true'
runs-on: ubuntu-24.04
permissions:
contents: read
# 180, and the reason is that 120 was measured on a quiet runner.
#
# Run 34583112120 (PR #140, full matrix) had legs of 40.8 to 90.4 min on an
# uncontended queue, and 120 looked like 33% headroom. Under contention it is
# not headroom at all. Run 34632419075 attempt 3 is the receipt: its six
# integrity legs sat QUEUED for 13 hours, started at 07:40Z, and five of them
# ended at 09:40:26Z — exactly 120 minutes, to the second, which is this
# timeout killing them. GitHub renders a timeout kill as CANCELLED, so it does
# not even read as a failure. Only 3.9 finished, at 93 min. On the same night
# main's own legs took 57 to 90 min with three workflows sharing the runners.
#
# So the old value failed its own matrix precisely when the queue was busy,
# which is exactly when it matters. A timeout is there to turn a hang into a
# failure, not to turn a slow queue into one.
#
# Lower it when the regex prefilter work lands and the real numbers drop.
timeout-minutes: 360
# One version was tested while setup.py claimed five. A support claim backed by
# no run is the same class of defect as a published number with no generator:
# believable, unverified, and only discovered by the user it breaks.
strategy:
fail-fast: false
matrix:
# 3.14 added 2026-09-08 (Fugu Gate-C addendum): the independent A1 review of
# the v0.5.6 repair ran on CPython 3.14 with Pillow 12.3.0, and J1 is
# specifically about Pillow frame-iterator behaviour. Five green lanes that
# EXCLUDE the graded interpreter prove compatibility, not reproduction.
# 3.14 is fully blocking here — never informational, never continue-on-error.
python-version: ["3.9", "3.10", "3.11", "3.12", "3.13", "3.14"]
# Each interpreter runs the full suite as jobs that together are
# exactly the suite (tests/ci_shard.py), because the slowest
# interpreter's single job outgrew GitHub's six-hour job limit.
# Two parts each; 3.12 runs four, the two extra parts added below,
# because its parts are the longest jobs in the run and every pull
# request waits for the slowest one.
shard: ["1", "2"]
include:
- python-version: "3.12"
shard: "3"
- python-version: "3.12"
shard: "4"
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: ${{ matrix.python-version }}
# Native decoders/OCR. Without these the v0.5.6 media tests cannot run at all,
# and a decoder that is missing turns a real finding into silence.
- name: Install native media tools (Tesseract, zbar, ffmpeg)
# A stalled mirror used to hold a runner for the whole job timeout. Each command is
# capped, and a retry against the same stuck mirror never passes, so every attempt
# after the first moves to the main Ubuntu archive before it tries again. The runner
# picks its mirror from apt-mirrors.txt, so that list is what gets rewritten. Three
# stalled attempts of 200 s each plus overhead run past 10 minutes; 12 lets the
# third one finish before the step cap cuts in.
timeout-minutes: 12
run: |
mirror_lines() {
grep -hE '^(URIs:|deb )' /etc/apt/sources.list /etc/apt/sources.list.d/*.sources 2>/dev/null || true
cat /etc/apt/apt-mirrors.txt 2>/dev/null || true
}
APT_OPTS="-o Acquire::http::Timeout=30 -o Acquire::Retries=2"
for attempt in 1 2 3; do
if [ "$attempt" -gt 1 ]; then
# An install stopped by the timeout leaves dpkg half-configured; repair it so the retry can run.
sudo timeout 120 dpkg --configure -a >/dev/null 2>&1 || true
echo "apt mirror before:"; mirror_lines
printf 'http://archive.ubuntu.com/ubuntu/\tpriority:1\nhttp://security.ubuntu.com/ubuntu/\tpriority:2\n' \
| sudo tee /etc/apt/apt-mirrors.txt >/dev/null
sudo sed -i 's|azure\.archive\.ubuntu\.com|archive.ubuntu.com|g' /etc/apt/sources.list /etc/apt/sources.list.d/*.sources 2>/dev/null || true
echo "apt mirror after:"; mirror_lines
fi
if sudo timeout 200 apt-get $APT_OPTS update -qq \
&& sudo timeout 200 apt-get $APT_OPTS install -y -qq tesseract-ocr libzbar0 ffmpeg; then
exit 0
fi
echo "apt attempt $attempt did not finish; retrying" >&2
done
exit 1
- name: Install package + dev + media deps
run: |
# PINNED PER INTERPRETER, not unpinned. pip 26.1 onward declares
# Requires-Python >= 3.10, so `pip==26.2.1` has no distribution for the
# 3.9 leg and that leg failed on this exact line. 26.0.1 is the newest
# pip 3.9 can take, read off the resolver's own list in the failing
# log rather than guessed. Markers rather than a shell branch: pip
# evaluates them and ignores the one that does not apply, so both pins
# stay visible as pins.
python -m pip install --upgrade \
'pip==26.2.1; python_version >= "3.10"' \
'pip==26.0.1; python_version < "3.10"'
# pytest stays unpinned in this fallback. The matrix spans Python 3.9 to 3.14 and a
# single exact pin cannot satisfy both ends: CI currently resolves 8.4.2 on the
# old legs and 9.1.1 on the new ones. Pillow is pinned per leg two lines below.
pip install -e ".[dev,media]" || { pip install -e . && pip install pytest "pytesseract==0.3.13" "PyPDF2==3.0.1" "pyzbar==0.1.9" Pillow; }
# Pin the reviewer's resolved stack rather than trusting free resolution.
# Pillow 12.3.0 requires >=3.10, so 3.9 necessarily takes 11.3.0 — that lane
# therefore differs from the graded stack on the exact library J1 concerns.
# It stays FULLY BLOCKING anyway: no exception is pre-authorised, and any
# difference there gets investigated before it is classified.
if [ "${{ matrix.python-version }}" = "3.9" ]; then
pip install "Pillow==11.3.0"
else
pip install "Pillow==12.3.0"
fi
pip install "pytesseract==0.3.13" "PyPDF2==3.0.1" "pyzbar==0.1.9"
# A version receipt, printed per job. Interpreter-dependent resolution can change
# frame-reading behaviour, which is the J1 surface — so the versions that actually
# ran are recorded, not assumed.
- name: Resolved environment receipt
run: |
echo "python : $(python -V 2>&1)"
echo "pip : $(pip -V)"
echo "pytest : $(pytest --version 2>&1 | head -1)"
python - <<'PYEOF'
import importlib.metadata as md
for pkg in ("Pillow", "pytesseract", "PyPDF2", "pyzbar"):
try:
print(f"{pkg:12s}: {md.version(pkg)}")
except Exception as exc:
raise SystemExit(f"MISSING media dependency {pkg}: {exc}")
PYEOF
echo "tesseract: $(tesseract --version 2>&1 | head -1)"
echo "ffmpeg : $(ffmpeg -version 2>&1 | head -1)"
echo "ffprobe : $(ffprobe -version 2>&1 | head -1)"
# Importability is not operation. A decoder that imports but never decodes
# reports a real finding as noise, so the modules are imported AND the two
# committed QR fixtures are actually decoded, each required non-empty.
- name: Media dependency preflight (import + PATH + real QR decode)
run: |
python - <<'PYEOF'
import shutil, sys
import PIL, pytesseract, PyPDF2 # noqa: F401
import pyzbar.pyzbar as zb
from PIL import Image
for tool in ("tesseract", "ffmpeg", "ffprobe"):
if shutil.which(tool) is None:
raise SystemExit(f"NATIVE TOOL MISSING on PATH: {tool}")
for fixture in ("tests/fixtures/v056/qr-ordinary.png",
"tests/fixtures/v056/qr-injection.png"):
with Image.open(fixture) as img:
codes = zb.decode(img.convert("RGB"))
if not codes or not codes[0].data:
raise SystemExit(f"QR CONTROL FAILED: {fixture} decoded to nothing "
f"-- zbar is present but not operational")
print(f"QR OK {fixture}: {len(codes)} code(s), {len(codes[0].data)} bytes")
print("media preflight OK")
PYEOF
- name: Pattern integrity tests (no dupes / id / category / shape)
run: pytest tests/test_pattern_integrity.py -v
- name: Dogfood regression tests (May 27 incident)
run: pytest tests/test_dogfood_bugs.py -v
- name: False-positive regression gate (Jun 6 incident — clean text must not block)
run: pytest tests/test_false_positives.py -v
- name: Customer-zero real-world suite (attacks block, normal usage passes)
run: python test_customer_zero.py
# v0.5.6 round-8 acceptance, as a DEDICATED BLOCKING step ahead of the full
# suite. These are the cells that discriminate the J1/J2 repair: 10 of them
# fail on the previous candidate. Exactly 19 must pass and NOTHING may skip --
# a skip here would let this workflow report success while omitting the very
# evidence the independent GO rests on.
- name: v0.5.6 round-8 acceptance (J1/J2) — 19 passed, zero skips
run: |
pytest tests/test_v056_round8.py -q -p no:cacheprovider \
| tee /tmp/round8.log
CODE=${PIPESTATUS[0]}
grep -qE "^19 passed" /tmp/round8.log || {
echo "ROUND-8 GATE INVALID: expected exactly '19 passed'"; exit 1; }
if grep -qE "[0-9]+ (skipped|error|failed)" /tmp/round8.log; then
echo "ROUND-8 GATE INVALID: a skip/error/failure is present"; exit 1; fi
exit "$CODE"
# The boundary harness tripwire, as a blocking step in both lanes so the
# fast lane still mirrors the matrix step for step. The harness mirrors
# the records the proxy writes, and this file fails when the two drift
# apart. It went red when revoking became a durable record and nothing
# ran it for three days. Only the three tests that need no local review
# material run here. The rest of the boundary suite reads scenario and
# review files that live outside the repository and runs nightly on the
# machine that holds them. Exactly 3 must pass and nothing may skip.
- name: Boundary harness tripwire, 3 passed and zero skips
run: |
pytest -q -p no:cacheprovider \
"gauntlet/boundary/tests/test_upstream_closed_mirrors_the_product.py::test_the_product_records_two_of_the_four_and_not_the_other_two" \
"gauntlet/boundary/tests/test_upstream_closed_mirrors_the_product.py::test_the_upstreams_list_changed_is_recorded_as_the_product_records_it" \
"gauntlet/boundary/tests/test_upstream_closed_mirrors_the_product.py::test_no_revoke_record_without_the_upstreams_list_changed" \
| tee /tmp/boundary_tripwire.log
CODE=${PIPESTATUS[0]}
grep -qE "^3 passed" /tmp/boundary_tripwire.log || {
echo "TRIPWIRE GATE INVALID: expected exactly '3 passed'"; exit 1; }
if grep -qE "[0-9]+ (skipped|error|failed|deselected)" /tmp/boundary_tripwire.log; then
echo "TRIPWIRE GATE INVALID: a skip/error/failure is present"; exit 1; fi
exit "$CODE"
# The receipts library's own tests live inside the package
# (sunglasses/receipts/tests), which pytest.ini's testpaths does not name,
# so no bare `pytest` ever collected them. Five of the seven files stopped
# importing on 2026-10-21 and nothing noticed. Named here as a blocking step
# in both lanes so the fast lane still mirrors the matrix step for step.
# At least one test must pass and nothing may skip, error, fail or be
# deselected; a count is not pinned, so adding a test needs no edit here.
# tests/test_ci_gate.py fails if a test_*.py under sunglasses/ is outside
# every path the gate runs.
- name: Receipts library tests (sunglasses/receipts/tests), zero skips
run: |
pytest -q -p no:cacheprovider sunglasses/receipts/tests \
| tee /tmp/receipts_lib.log
CODE=${PIPESTATUS[0]}
grep -qE "^[0-9]+ passed" /tmp/receipts_lib.log || {
echo "RECEIPTS GATE INVALID: no passing test was reported"; exit 1; }
if grep -qE "[0-9]+ (skipped|error|failed|deselected)" /tmp/receipts_lib.log; then
echo "RECEIPTS GATE INVALID: a skip/error/failure is present"; exit 1; fi
exit "$CODE"
# Full suite added 2026-07-10 (Strange audit): the 4 targeted gates above
# only cover ~4 test families; major families ran local-only until now.
- name: Full test suite
# --durations=0 (2026-09-17, second reading). --durations=25 was
# added this morning and answered the first question: the 3.12 leg
# is the slowest in 7 of 8 full-matrix runs (median 135 min against
# 61-91 for every other interpreter) and the excess is a fixed ~94s
# wait, not slow work -- run 35291331593 put the same commit at
# 8248s on 3.12 and 4459s on 3.13, and 3.12's rows are quantised at
# 94.1-94.9s on the `setup` AND `call` of tiny unrelated tests. A
# truncated list cannot COUNT those waits, because 22 of its 25 rows
# are already sitting at the ceiling; the estimate 3789s / 94.3s ~= 40
# is arithmetic, not a measurement. =0 prints every row, so the next
# full-matrix run says exactly how many tests wait and which ones,
# which is the difference between a suspicion and a fix. The cost is
# log lines on a job that already runs for over two hours.
env:
CI_SHARD: ${{ matrix.shard }}/${{ matrix.python-version == '3.12' && 4 || 2 }}
run: pytest -q -rfEs --durations=0 ${{ needs.classify.outputs.quick == 'true' && '-m "not full_matrix"' || '' }}
coverage:
# The single check merge automation may rely on. always() so a skipped or
# failed upstream job still produces a verdict here instead of silence.
needs: [classify, fast, integrity]
if: always()
runs-on: ubuntu-24.04
# No per-job permissions here, deliberately. `coverage` is THE gate and
# its key set is an ASTRA reviewed contract (tests/test_ci_gate.py,
# COVERAGE_JOB_KEYS, round 3 R3-2) that exists so an unreviewed shape
# cannot change what the gate means. The top-level `permissions: read-all`
# already grants this job the minimum, so a block here would buy nothing
# and would edit a shape somebody approved.
timeout-minutes: 5
env:
CLASSIFY: ${{ needs.classify.result }}
FULL: ${{ needs.classify.outputs.full }}
FAST: ${{ needs.fast.result }}
INTEGRITY: ${{ needs.integrity.result }}
steps:
- name: Require the work the change needed
run: |
echo "classify=$CLASSIFY full=$FULL fast=$FAST integrity=$INTEGRITY"
[ "$CLASSIFY" = "success" ] || { echo "COVERAGE FAIL: classify did not succeed"; exit 1; }
[ "$FAST" = "success" ] || { echo "COVERAGE FAIL: fast lane did not succeed"; exit 1; }
if [ "$FULL" = "true" ]; then
[ "$INTEGRITY" = "success" ] || { echo "COVERAGE FAIL: full matrix required and not green ($INTEGRITY)"; exit 1; }
echo "COVERAGE OK: full matrix required and green"
else
[ "$FULL" = "false" ] || { echo "COVERAGE FAIL: classify output is neither true nor false ($FULL)"; exit 1; }
echo "COVERAGE OK: documentation-only change, fast lane green"
fi
certify:
# THE gate. `coverage` answers "did this change get the work it needed",
# which is the right question for a MERGE and the wrong one for a RELEASE:
# a documentation-only change legitimately skips the matrix, and a tree
# verified only by the docs lane must never be publishable.
#
# So certify answers both, separately. It is the single required status
# context for merging, and it records in its own log whether the FULL
# matrix actually ran, which is what the publisher checks before uploading.
#
# Built 2026-09-10 after a system audit found the release path had no
# mechanical gate at all: PyPI could be uploaded from a tree whose CI had
# skipped, been cancelled, or never run. A missing leg is not a pass.
needs: [classify, fast, integrity, coverage]
if: always()
runs-on: ubuntu-24.04
permissions:
contents: read
timeout-minutes: 5
env:
CLASSIFY: ${{ needs.classify.result }}
FULL: ${{ needs.classify.outputs.full }}
FAST: ${{ needs.fast.result }}
INTEGRITY: ${{ needs.integrity.result }}
COVERAGE: ${{ needs.coverage.result }}
steps:
- name: Every required leg ran and succeeded
run: |
echo "classify=$CLASSIFY full=$FULL fast=$FAST integrity=$INTEGRITY coverage=$COVERAGE"
# An absent result is the failure mode this job exists for. A leg that
# was skipped, cancelled, or never reported reads as anything but
# "success", and none of those are evidence that the work was done.
fail() { echo "CERTIFY FAIL: $1"; exit 1; }
[ -n "$CLASSIFY" ] || fail "classify produced no result at all"
[ -n "$FAST" ] || fail "fast produced no result at all"
[ -n "$COVERAGE" ] || fail "coverage produced no result at all"
[ "$CLASSIFY" = "success" ] || fail "classify=$CLASSIFY"
[ "$FAST" = "success" ] || fail "fast=$FAST"
[ "$COVERAGE" = "success" ] || fail "coverage=$COVERAGE"
case "$FULL" in
true)
[ "$INTEGRITY" = "success" ] || fail "full matrix required, integrity=$INTEGRITY"
echo "CERTIFY LEVEL: full"
echo "CERTIFY OK: every leg ran and succeeded on $GITHUB_SHA"
;;
false)
# The docs lane. Merging is fine; publishing from this tree is not.
#
# This comment used to say the publisher "refuses on the absence of
# CERTIFY LEVEL: full". Nothing reads that string -- not
# sunglasses-publish.sh, not any tool or ship script (grepped
# 2026-09-21, zero hits outside this file). The refusal is real but
# it comes from somewhere else: sunglasses-publish.sh runs
# scripts/require_release_certification.py on the release commit and
# blocks on its EXIT CODE, and that script asks the API whether the
# full matrix ran green on that exact sha.
#
# Naming a guard that does not exist is worse than naming none: the
# next person to change this lane protects the wrong string and
# leaves the real gate unexamined. The lines below are for a HUMAN
# reading the run log; they are not a machine contract.
[ "$INTEGRITY" = "skipped" ] || fail "docs lane but integrity=$INTEGRITY (expected skipped)"
echo "CERTIFY LEVEL: docs"
echo "CERTIFY OK for merge: documentation-only change, fast lane green on $GITHUB_SHA"
echo "NOT RELEASE CERTIFIED: the full matrix did not run on this tree"
;;
*)
fail "classify output is neither true nor false ($FULL) — refusing to guess"
;;
esac