fix(freeipa,dns): leave out replicas FreeIPA does not list as DNS servers; one forwarder rule (follow-up to #73) by kjelly · Pull Request #92 · kjelly/pilot

kjelly · 2026-10-14T13:00:15Z

Follow-up to #73 (merged as f9fb701). Six rounds; the first two reviews: the second review of #73's head 5e48d67 (round 1, candidate 01a64f4), and a review of this PR's head 44e2d20 (round 2, candidate 09ded7e, then 5750f2a after my own adversarial review).

⚠️ Breaking change

freeipa-server-apply.yml now sets two lists to ipa_dns_forwarders / freeipa_dns_forwarders on every run:

the realm's global forwarders (ipa dnsconfig-mod);
the primary's own forwarders (ipa dnsserver-mod <fqdn>). FreeIPA stores the install-time --forwarder there, and named prefers it over the global list.

An explicitly empty value ("", [], null) clears both on an existing realm. FreeIPA's named then answers only its own zones. Before, an empty value skipped this step, and a changed list never reached the server's install-time forwarders.

freeipa-server-replica-apply.yml likewise sets a replica's own forwarders. Unset is the default; it clears them, and the replica uses the global list.

Leaving the variable unset on the primary still means 8.8.8.8. Documented in freeipa-server.md v1.9, freeipa-server-replica.md v1.2 and group_vars/freeipa.example.yml.

Current evidence (candidate `b82a0c9`)

b82a0c9 (tree b733b150) is this PR with the round 6 fix (dee1fe8), merged with main afeacbd (724cc45, a webhook fix in internal/outbound, plus an evidence record). CI passed on it (run 38880469725). The head adds only the evidence record and doc summaries.

Record: docs/evidence/freeipa-replica-dns-provider/2026-10-16-b82a0c9.md. It repeats every check of the cdcd39b record on this candidate, with the same 44e2d20 and main f9fb701 controls and the no-fix diagnose binary:

Topologies: a new six-VM topology (A), and a four-VM one (B) for the consumer cases. L3–L6 pass on both; L5 20/0, 16/0, 7/0 (A) and 20/0, 7/0 (B); dns.md 16 pass / 4 n/a. All three changed kinit tasks ran in L4 and L6 with the vault password.
fix(diagnose): name every failed or unknown DNS fact; refuse privileged principals (follow-up to #66) #91's cases: the tier refuses the client, a trailing-dot FQDN, K/M/k principals, and the stub and FreeIPA BIND as the first hop. Each diagnose server now uses its checkout as --dir, so the supported layer reports the component contracts. In the cdcd39b run it was an empty directory.
Round 2 (R2.1–R2.6) and round 1: a replica under another name, a stopped named, primary and replica forwarders, the forwarder and install-time gates, the unpromoted replica, site.yml, and promotion. Every result matches the earlier records.
Special-character admin passwords, live. The throwaway values have $, backtick and $(...) commands, one with quotes and one without. Each control fails, and with the quote-free value the touch commands in the password run as root. b82a0c9 succeeds, the rerun gives changed=0, and nothing in the password runs.
- (a) Single-host vm-target inventory (no freeipa-server group; the self-register block is skipped), replica playbook on an existing DNS replica. c6bcfe9 fails at the forwarder kinit, and the commands run on the replica.
- (b) Topology inventory (the self-register block runs delegated to the primary), replica playbook. 3dbc0ef fails at the self-register kinit, and the commands run on the primary.
- (c) The primary, freeipa-server-apply.yml --tags freeipa-verify. Of the 7 command tasks there and in playbooks/apply/tasks/ that read an admin or Directory Manager password, this tag selects only the dnsconfig-mod kinit. 3dbc0ef fails at that kinit, and the commands run on the primary.
- The password was changed back to the vault value afterwards. The other shell kinits and ldapmodify -w still on main (bug(playbooks): 33 個 task 把密碼插進 shell/command 字串，密碼中的 $、反引號、引號會被改寫或執行 #118) would still break with such a password, so the full chain was not run with one. A scan of every log and diagnose result found no password.
Behavioural test: TestRegression_FreeIPAKinitTasksPassPasswordUnchanged passed all 18 subtests locally, with ansible-core 2.19.2 and -v. CI's Go test job installs no Ansible, but its runner image (ubuntu-24.04 20261012.334) lists Ansible 2.21.5, so the test may run in CI rather than skip; go test without -v does not show which. Only the local run counts as evidence.
Run conditions and metadata: a clean checkout of the committed candidate, with every input outside it. The docs/actual-run-evidence.md §3 fields were captured during the run. The record lists the git blob of all 688 execution files, now including contracts/ and ansible_callback/, and says which directories count.
Local checks from the same checkout: go test -race 57 ok, golangci-lint 2.12.1 0 issues, playbook-lint, spec lint 0 findings.

The earlier records (cdcd39b, b02b5cb, 287cf79×3, 5750f2a, 09ded7e, 01a64f4) are marked superseded. The round sections below describe the changes; the record above has their verification.

Round 6: the self-register and dnsconfig-mod `kinit`s (review of `3dbc0ef`)

[B1] fix (dee1fe8). Two more tasks gave the admin password to kinit in a shell command line: the replica self-register kinit, which runs whenever the inventory has a freeipa-server group, and the primary's dnsconfig-mod kinit. Both now run kinit through argv with the password on stdin. The self-register KRB5CCNAME comes from the task's environment.
Regression test. TestRegression_FreeIPAKinitTasksPassPasswordUnchanged runs all three real tasks with ansible-playbook and a fake kinit, and checks that self-register uses its dedicated cache.
Lint. The allowlist counts tasks per <file>::<task name>. The name pattern adds _pass, pwd, token, api_key and psk, and excludes _file, _path and _dir. The allowlist stays at 33 entries; bug(playbooks): 33 個 task 把密碼插進 shell/command 字串，密碼中的 $、反引號、引號會被改寫或執行 #118 is updated.
Docs. AGENTS.md §4.5 point 15, a §6 line, and changelog v1.53.
Merge. main afeacbd touches no file on this PR's DNS or diagnose path, but it changes the pilot binary.

Round 5: the replica forwarder `kinit` (review of `c6bcfe9`; candidate `cdcd39b`, superseded by round 6)

[P2] fix. The replica forwarder task added in this PR gave the password to kinit as printf %s "{{ ipa_admin_password }}" | kinit … in bash. Ansible puts the value into the command text before the shell parses it, so $, backticks and $(...) were expanded or run. It now runs argv: [kinit, "admin@{{ ipa_realm }}"] with the password on stdin; no_log and the private credential cache are unchanged.
Regression test. TestRegression_ReplicaForwarderKinitPassesPasswordUnchanged runs the real task with ansible-playbook and a fake kinit. On c6bcfe9, four of its six passwords fail: kinit gets demo, ab or xy, the touch commands run, or a password with quotes fails the play.
Same pattern elsewhere (AGENTS §5.10). 33 tasks already on main put a password-like variable into shell or free-form command text. The new TestRegression_NoSecretInCommandText fails for any new one. The 33 are a ratchet allowlist that also fails when an entry stops matching; migrating them is bug(playbooks): 33 個 task 把密碼插進 shell/command 字串，密碼中的 $、反引號、引號會被改寫或執行 #118.
Merge. main f98327b (fix(access): parallelize gateway queries and clarify API timeouts #112, ci: raise the Go test timeout to 30m #116) touches no file on this PR's DNS or diagnose path, but it changes the pilot binary, so every check was re-run.

Round 4: merged with `main` again; left-out replicas in the paths (candidate `b02b5cb`, superseded by round 5)

Merge. main cc38f58 changed files on this PR's tested path: the private-cache task files the FreeIPA playbooks include, and internal/spec/generator.go. It had no conflicts.
Fix. BuildDNSArchPaths printed a left-out replica as "FreeIPA DNS replica … authoritative for ". It now says that replica is left out of the DNS paths. TestBuildDNSArchPaths_LeftOutReplicaIsNotAuthoritative fails without the fix.

Round 2

A replica that answers is kept. tasks/freeipa-dns-providers-live.yml keeps a replica whose own +norec NS answer is NOERROR with aa, even when its inventory name is not in FreeIPA's NS records. Before, a replica installed under another name was dropped silently.
- Probes use +tries=2.
- "No FreeIPA server answered" is reported.
- The WARNING gives four remedies.
- pilot_diagnose_dns_architecture follows the same rule.
The resolver check blocks only what must be blocked. The new tasks/freeipa-dns-client-nameserver-check.yml asks each nameserver for A and AAAA. It fails only when:
- a server answers without an address;
- a FreeIPA server that FreeIPA does not list does not answer;
- no server answers at all.
A server that is down now gets a WARNING with a remedy for its role, and stays in the list: a listed FreeIPA server, a dns tier host, or an explicit entry. Only FreeIPA servers are asked with +norec. The tier's G5 still fails closed when a listed FreeIPA server does not answer; that is documented in the dns runbook.
Address check. Python ipaddress checks forwarders on the controller, and Go net/netip in the diagnose model. TestNormalizeForwardersMatchesPlaybookGate now covers 15 more inputs.
- "<address> port <n>" is accepted (checked live with ipa dnsconfig-mod and dnsserver-mod on 4.13.4).
- Unspecified, multicast and zoned addresses are refused, as is a port outside 1–65535.
- The installers take plain addresses only, so a pre_tasks gate stops a port form before any change when the run would install.
Adversarial-review fixes (5750f2a).
- pilot_diagnose_dns_architecture and its runbook no longer call per-server forwarders unmanaged. A mismatch now says to re-apply.
- The Go mirror uses only the replica's own NS answer, as the playbook does.
- IPv4-mapped addresses are judged as the IPv4 address they map, on both sides (::ffff:0.0.0.0 is refused).
- The WARNING says whether FreeIPA lists the server, or that no FreeIPA server answered.
Visibility. dns-apply.yml prints DNS — summary: FreeIPA servers the stubs use: the effective servers, the ones left out, and who answered.

Round 1 (`01a64f4`, unchanged)

A replica that is listed in the inventory but not promoted no longer breaks the site-wide deploy, and is no longer written as a dead nameserver.
One forwarder rule is shared by the primary and the replicas.
Docs.

Verification (round 2)

History: these results are for 5750f2a, whose record is superseded. Every check was repeated on the current candidate; see Current evidence.

Records: docs/evidence/freeipa-replica-dns-provider/2026-10-15-09ded7e.md has the control comparisons against 44e2d20. …/2026-10-15-5750f2a.md has the final candidate, re-run on a new fresh pr92- topology.

Local checks (clean checkout of 5750f2a; also green on 09ded7e):
- go test -race -count=1 -timeout 60m ./...: 57 packages ok.
- golangci-lint 2.12.1: 0 issues.
- make playbook-lint: pass.
- Spec lint: 20 / 7 / 16 / 20 rows, 0 findings.
Fail-before / pass-after:
- TestRegression_ResolverNameserverCheck has 10 cases from real dig captures. The old gate fails the down-replica, IPv6-only and explicit-timeout cases.
- TestDNSArchUnregisteredReplicas_KeepAReplicaThatAnswersItself fails on 44e2d20.
- 10 of the new forwarder cases fail on 44e2d20.
- ::ffff:0.0.0.0 fails on 09ded7e.
Fresh pr92- topology, run twice: 09ded7e, then 5750f2a on new VMs (6 VMs: FreeIPA 4.13.4 primary and two replicas, one more replica listed but never promoted, an unbound tier, a consumer):
- L3 failed=0; L4 failed=0; L6 changed=0.
- L5 freeipa-server 20/0, freeipa-server-replica 16/0, freeipa-dns-client 7/0, dns 16 pass / 4 n/a.
Targeted checks:

Case	`44e2d20`	`09ded7e` / `5750f2a`
Replica whose inventory name differs from FreeIPA's	left out (WARNING)	kept; stub `.4 .7 .8`
Listed replica's named stopped, consumer	rc=2, nothing written	rc=0, WARNING `check named on pr92-r2`
Listed replica's named stopped, tier	—	G5 rc=2 (unchanged, documented)
Primary `freeipa_dns_forwarders: []` on the existing realm	changed=0, `8.8.8.8` stays; diagnose `mismatch`	changed=2: global and server forwarders empty; rerun changed=0; diagnose `match`
`"1.1.1.1 port 53 9.9.9.9"` on the existing realm	refused	applied to both lists; rerun changed=0
Bad values (`"8.8.8.8,1.1.1.1"`, `port 0`, `port 65536`, `0.0.0.0`, `224.0.0.1`, `:::`, `fe80::1%eth0`, `{}`)	—	rc=2, changed=0
Port form before installing on the unpromoted replica	—	rc=2, changed=0, nothing installed

Diagnose on the final state: complete, 12/12 match, finding only for the unpromoted replica.

Round 3: merged with `main` (#91), candidate `287cf79` (superseded by round 4)

#91 merged into main as 306c444 first, so this PR resolved the conflict, as agreed. 287cf79 (tree 1cbaac4f) is fde3ce4 merged with 306c444. Record: docs/evidence/freeipa-replica-dns-provider/2026-10-15-287cf79.md.

Conflicts in four files, each resolved by keeping both sides: the MCP diagnose file, the report, the test file and the diagnose runbook. The record lists them.
Semantic conflict. fix(diagnose): name every failed or unknown DNS fact; refuse privileged principals (follow-up to #66) #91's provider comparison now skips left-out replicas (… || !p.DNSProvider || p.LeftOut || …).
- New test TestBuildDNSArchDifferences_LeftOutReplicaIsNotComparedWithTheReference fails without that change.
- fix(diagnose): name every failed or unknown DNS fact; refuse privileged principals (follow-up to #66) #91's TestBuildDNSArchStatus_UnknownConfigAndComparisonsAreNamed now expects this PR's subject, (ipa dnsserver-mod).
- The tier's authority probes of left-out replicas were already dropped by this PR's side, and the live run shows no freeipa_dns_not_authoritative_from_tier.
Other execution files. The other ten files that differ from 5750f2a are main's blobs, unchanged. No playbook, task file, fixture, contract or spec changed.
Local checks (clean checkout of the committed 287cf79):
- go test -race -count=1 -timeout 60m ./...: 57 packages ok.
- golangci-lint 2.12.1: 0 issues.
- make playbook-lint: pass.
- Spec lint: 20 / 7 / 16 / 20 rows, 0 findings.
New six-VM pr92- topology, run from that checkout, with the §3 metadata captured during the run (run IDs, UTC window, base image sha256, package versions, input digests):
- L3 failed=0; L4 failed=0; L6 changed=0.
- L5: 20/0, 16/0, 7/0; dns.md 16 pass / 4 n/a.
- The targeted checks above gave the same results.
- Diagnose: complete, 14/14 match, with only freeipa_replica_not_dns_server for the unpromoted replica. A control binary, the same tree without the left-out skip, gave partial with comparison_unknown for that replica.
- The stub-first-hop client has no false first-hop finding.
fix(diagnose): name every failed or unknown DNS fact; refuse privileged principals (follow-up to #66) #91's other cases (tier refuses the client, trailing-dot FQDN, K/M/k) were not re-run on 287cf79.

🤖 Generated with Claude Code

…vers; one forwarder rule Second review of #73 (head 5e48d67): - P2: a replica with freeipa_setup_dns unset counts as a DNS provider, but freeipa-server-replica is a day-2 playbook outside site.yml, so an inventory lists it before it is promoted; a site-wide deploy then failed at dns.md G5, and a consumer got a dead nameserver silently. New tasks/freeipa-dns-providers-live.yml asks FreeIPA itself (the zone's NS records, +norec) which servers run its DNS. A replica that leaves freeipa_setup_dns unset and is not listed is left out of the tier's stubs and the consumer's nameservers with a WARNING naming it and the two remedies; an explicit true stays in and fails closed at G5 or at the resolver. dns-apply.yml asks after dig is installed; the G5 and check-mode messages now name the replica case. The resolver installs dig first, asks the same, and then asks every nameserver it will write (FreeIPA +norec, tier and explicit servers recursively) for the FreeIPA server's FQDN before writing anything; one that does not answer fails the run with the list and the remedies, before any change. pilot_diagnose_dns_architecture follows the same NS records: such a replica is freeipa_replica_not_dns_server and the effective stubs and resolver lists leave it out. - P3: the primary and a replica now read forwarders identically (tasks/freeipa-dns-forwarders.yml picks ipa_dns_forwarders if defined, else freeipa_dns_forwarders if defined, else the caller's default: 8.8.8.8 on the primary, [] on a replica). An explicit empty value is no forwarders on both; before, the primary turned "", [] and null into 8.8.8.8. Every forwarder must be an IPv4/IPv6 address, checked before any install ("8.8.8.8,1.1.1.1", "1.1.1.1 port 53", {}, a number and an unclosed "[..." used to pass or crash). The Go model mirrors it; TestNormalizeForwardersMatchesPlaybookGate runs the task on 21 inputs and compares. - P3: stale "replicas default false / opt-in" comments and docs swept; freeipa_dns_server_ips documented as every server the tier stubs to; the dns runbook gate table covers the replica cases; the forwarder docs say exactly what both playbooks do. Co-Authored-By: Claude Opus 5.5 <[email protected]>

New record 2026-10-14-01a64f4.md: local checks; the topology test on four fresh VMs (L3/L4 failed=0, L5 pass, L6 changed=0); a replica listed before it is promoted (main fails at G5 and writes it as a nameserver; the candidate leaves it out with a warning, site.yml passes, an explicit true fails closed before any change; after promotion it is included and reruns are changed=0); the forwarder gate refusing bad values on the primary. docs/runbooks/dns.md §0.5 now describes this run's target and inventory. Co-Authored-By: Claude Opus 5.5 <[email protected]>

kjelly · 2026-10-14T13:00:40Z

這是 #73（已合併為 f9fb701）的後續修正，處理二次審查對 5e48d67 提出的問題。head 44e2d20，candidate 01a64f4（tree 1475bb8d）。

[P2] 還沒 promote 的 replica 會讓全站部署失敗，並被寫成無效的 nameserver
- 修正：改問 FreeIPA 的 NS 紀錄（+norec）。沒被列出、freeipa_setup_dns 也沒設的 replica 會被排除，並印出指名它的 WARNING 和兩種補救方式；明確設成 true 時，仍在 G5 或 resolver 階段 fail closed。resolver 寫入前會先逐一詢問每個 nameserver。
- main 上實測：dns 在 G5 失敗，client 把 replica 寫成 nameserver（rc=0）。
- candidate 實測：site.yml 五台都是 failed=0；promote 之後 replica 被納入，重跑 changed=0。
[P3] forwarder：primary 與 replica 改用同一條規則；每個值都必須是 IPv4 或 IPv6 位址，在任何安裝之前檢查。實測 "8.8.8.8,1.1.1.1"、"1.1.1.1 port 53"、{} 都被拒絕（rc=2、changed=0）。
[P3] 文件：更新註解、freeipa_dns_server_ips 的說明、gate 表，以及 dns runbook §0.5 的事實快照。

驗證

go test -race 57 個 package 全部 ok；golangci-lint 0 issues；playbook-lint 與 spec lint 都通過。
4 台全新 VM 的 topology test：L3／L4 failed=0；L5 freeipa-server 20、freeipa-server-replica 16、dns 16（另有 4 列 N/A）、freeipa-dns-client 7；L6 changed=0。

詳見 docs/evidence/freeipa-replica-dns-provider/2026-10-14-01a64f4.md。

這個 PR 和 #91 會改到相同的檔案，後合併的那個負責解衝突。

… are down, clear empty forwarders Second review of #92 (the #73 follow-up): - tasks/freeipa-dns-providers-live.yml: a replica whose own +norec NS answer is NOERROR+aa stays in, whatever name the inventory gives it; before, a replica installed under another name was left out because its inventory FQDN was not in FreeIPA's NS records. dig uses +tries=2, and "no FreeIPA server answered" is printed. The WARNING lists the four remedies. pilot_diagnose_dns_architecture follows the same rule (its provider probe also uses +tries=2). - tasks/freeipa-dns-client-nameserver-check.yml (new): the resolver asks every nameserver for the FreeIPA FQDN's A and AAAA, and fails only for a server that answers without an address, a FreeIPA server FreeIPA does not list that does not answer, or no working server. A server that is only down now (registered FreeIPA server, dns tier, explicit entry) gets a WARNING with a role-specific remedy and stays in. +norec only for FreeIPA servers; an explicit 127.0.0.1 is asked recursively. - tasks/freeipa-dns-forwarders.yml: forwarders are checked with Python's ipaddress on the controller (and net/netip in internal/diagnose); "<address> port <n>" is accepted (ipa dnsconfig-mod takes it, checked on FreeIPA 4.13.4), unspecified/multicast/zoned addresses are refused. The installers take plain addresses only, so when a run would call ipa-server-install, ipa-replica-install or ipa-dns-install with a port form, a pre_tasks gate stops it before any change. - freeipa-server-apply.yml: every run sets the realm's global forwarders (ipa dnsconfig-mod) and the primary's own forwarders (ipa dnsserver-mod <fqdn>, which its named prefers over the global ones) to the list; an explicitly empty list clears both, on an existing realm too. Before, an empty list skipped this, and a changed list never reached the server's install-time forwarders. freeipa-server-replica-apply.yml does the same for a replica's own forwarders (empty by default: it then uses the global ones). Breaking change, documented in freeipa-server.md v1.9, freeipa-server-replica.md v1.2 and group_vars/freeipa.example.yml. - dns-apply.yml prints the FreeIPA servers the stubs use and the ones left out; G5's message names named on a listed server. Tests from real pr92 captures: TestRegression_ResolverNameserverCheck (10 cases), TestDNSArchUnregisteredReplicas_KeepAReplicaThatAnswersItself, and 15 new forwarder cases in TestNormalizeForwardersMatchesPlaybookGate. Co-Authored-By: Claude Opus 5.5 <[email protected]>

Co-Authored-By: Claude Opus 5.5 <[email protected]>

kjelly · 2026-10-14T20:25:38Z

第二輪 review 的問題已處理完，新 head 26e5cfb（candidate 5750f2a；5750f2a 之後的 commit 只加 evidence 與 dns runbook §0.5）。

review 項目 → 處理方式

[P3] 名稱對不上的可用 replica 被默默略過 → 改成 replica 自己 +norec 權威回答 zone（NOERROR+aa）就保留。WARNING 改寫，列出四個做法。diagnose 也採用同一條規則。
[P3] +tries=1 → 改成 +tries=2；沒有任何 FreeIPA server 回答時會明確印出訊息。
[P3] resolver 的 nameserver gate 重新設計（新的 tasks/freeipa-dns-client-nameserver-check.yml，同時問 A 與 AAAA）。
- 只有三種情況會擋下：有回答但沒有位址、FreeIPA 不承認也不回答的 FreeIPA server、沒有任何一台可用。
- 已登記但暫時不回答的 server 印 WARNING，並依角色給出做法。
- 只有 FreeIPA server 用 +norec。
- 測試用真實擷取的輸出，spec v1.4 與 runbook 已寫明。
[P3] 空 forwarders 不會收斂 → 每次 apply 用 ipa dnsconfig-mod 與 ipa dnsserver-mod <fqdn> 讓全域與 server 自己的 forwarders 都等於設定值，空值兩者都清掉。實跑時發現只清全域不夠：install 時寫進 server 的 8.8.8.8 會留著，而 named 優先用它；replica 也比照處理。這是破壞性變更，已寫進 freeipa-server.md v1.9、freeipa-server-replica.md v1.2、example 檔與 PR 描述。
[P3] 位址檢查 → Ansible 端用 Python ipaddress、Go 端用 netip。<IP> port N 已在 4.13.4 實測可用，安裝器不收這種寫法，所以 run 需要安裝時會在 pre_tasks 擋下。TestNormalizeForwardersMatchesPlaybookGate 新增 15 個案例。
[P3] 可見性 → dns-apply.yml 新增摘要，列出實際使用與被略過的 FreeIPA server。

自己的 adversarial review（09ded7e → 5750f2a）

diagnose 與其 runbook 仍說各 server 的 forwarders 不歸 pilot 管 → 改成提示重新 apply，NotManaged 也移除這一項。
Go 端判斷 replica「自己回答」時多接受了 SOA → 改成跟 playbook 一樣只看 NS。
::ffff:0.0.0.0 在 Go 端會過 → 兩邊都把 IPv4-mapped 位址當成它對應的 IPv4 位址判斷，並新增測試案例。
WARNING 文字會分清楚「FreeIPA 有列出這台」與「沒有任何 FreeIPA server 回答」。

驗證（docs/evidence/freeipa-replica-dns-provider/2026-10-15-09ded7e.md 是與 44e2d20 的對照，2026-10-15-5750f2a.md 是最終 candidate 的結果）

本機：go test -race 57 個 package ok；golangci-lint 0 issues；playbook-lint 與 spec lint 通過。三組新測試都確認在 44e2d20 會失敗。
全新 pr92- topology test，09ded7e 與 5750f2a 各在一組新建的 VM 上跑一次：L3／L4 failed=0；L5 20/16/7，dns.md 另外驗證 16 pass、4 n/a；L6 changed=0。
針對性實跑：名稱不符的 replica 保留；已登記 replica 停機時 consumer 只印 WARNING、tier 的 G5 照樣擋下；空 forwarders 收斂且重跑 changed=0，diagnose 全部 match；錯誤值與安裝前的 port 寫法都在變更前擋下。

另外觀察到：FreeIPA server 本機上，installer 的 zzz-ipa.conf 讓 resolv.conf 只剩 127.0.0.1。這在本 PR 之前就是如此，沒有修改。

Co-Authored-By: Claude Opus 5.5 <[email protected]>

…f answer, mapped addresses Review of 09ded7e: - pilot_diagnose_dns_architecture and its runbook still said pilot leaves each server's own forwarders alone; both playbooks now set them on every apply, so a mismatch means "re-apply", and the NotManaged entry is gone. - DNSArchUnregisteredReplicas counted a replica's own SOA answer too; tasks/freeipa-dns-providers-live.yml asks only NS, so the Go mirror now does the same. - An IPv4-mapped address is judged as the IPv4 address it maps on both sides (net/netip Unmap; Python's ipv4_mapped, which Python before 3.13 ignores for is_unspecified): ::ffff:0.0.0.0 and ::ffff:224.0.0.1 are refused by Go and by the playbook alike. - The resolver WARNING for a FreeIPA server that does not answer says whether FreeIPA lists it, or that no FreeIPA server answered. Co-Authored-By: Claude Opus 5.5 <[email protected]>

Co-Authored-By: Claude Opus 5.5 <[email protected]>

The 5750f2a run created a new set of six VMs with `topology up`; the §0.5 line still described the 09ded7e run's `topology reset`. Docs only. Co-Authored-By: Claude Opus 5.5 <[email protected]>

kjelly

重新審查 head a2f8e6811594994a8ae44210b941ab0114302e06：未發現新增功能阻擋，但驗證摘要尚未符合 repo 必要欄位，轉為草稿。

[P3] 最新 candidate 的 evidence 缺少可追溯 run metadata。 docs/evidence/freeipa-replica-dns-provider/2026-10-15-5750f2a.md 表示另建六台 VM，但只有 OS/package 版本，沒有 target image digest；Run 也只寫日期，沒有 UTC 時段或正式 run ID。docs/actual-run-evidence.md §3 明定 run ID、UTC 時間、target image digest 與適用的外部 state fingerprint 為必要欄位。

請從原 run 可追溯資料補足並連回 tested candidate/tree；若無法復原，再實跑取得。成功 raw artifacts 在摘要提交後刪除符合規則，並非本次阻擋原因。

本次核對 latest execution blob prefixes 與 candidate 一致；internal/diagnose 全包與 resolver nameserver 真實 Ansible cases 通過。另以 merge-tree 確認 #91/#92 在 MCP、report、tests、runbook 有實際衝突。兩者目前都未合併；之後先合併任一個，另一個須保留雙方修正、解衝突並驗證新 candidate。

The review of #92 found that the records lack the target image digest, the UTC window and run IDs (docs/actual-run-evidence.md §3). Add them to the 01a64f4, 09ded7e and 5750f2a records, read after the run from records that outlive the VMs: libvirt's QEMU log of each domain (VM start, reset, rollback and destroy times; the base image each overlay used), the verify evidence store (run IDs) and the base image files (sha256; last modified before the runs). The diagnose calls' audit records were deleted with the run's scratch directory, and the records say so. No execution file changes. Co-Authored-By: Claude Opus 5.5 <[email protected]>

kjelly · 2026-10-15T03:34:01Z

處理 2026-10-15 的 review：「驗證摘要缺少 image digest、UTC 時段及 run ID」。新 head fde3ce4，只改 evidence，執行檔沒有變更，candidate 仍是 5750f2a。

01a64f4、09ded7e、5750f2a 三份 record 各加了一節 Run metadata，內容是在 run 結束後，從 VM 刪除後仍保留的紀錄讀出來的，沒有從其他 run 推測：

UTC 時段與時間軸：從每個 VM 的 libvirt QEMU log 讀出啟動、reset、自動 rollback 與刪除的時間。例如 5750f2a 的 VM 從 2026-10-14 20:50:45Z 跑到 21:15:34Z。
run ID：每次 pilot verify 的 run ID 和結果數字，取自 pilot 的 verify evidence store（history.db），包含 L5 和 dns.md。
target image digest：每個 VM 的 overlay 所用的 base image 由 QEMU log 指出，sha256 是對 base image 檔算的。兩個檔案最後修改都在 2026-07-07，早於這幾次 run。
外部狀態：vault 只列 key 名稱（最後修改 2026-07-07）；dns.md C7、C8、C19 依賴公共 DNS，這部分沒有記錄。

還原不了的部分：diagnose 呼叫的 audit record 存在 run 的 scratch 目錄，已經跟著刪除，所以這些呼叫沒有 run ID。record 裡寫明了這一點。

kjelly

Review of head fde3ce4: keep draft pending integration with main.

The new evidence metadata addresses the previous run-ID/image-digest finding. Focused DNS architecture/forwarder race tests and real Ansible resolver-nameserver cases pass on this head.

PR #91 has now merged as 306c444. GitHub reports this PR CONFLICTING, and git merge-tree confirms four content conflicts: cmd/pilot/cmd/mcp_diagnose_dns_architecture.go, docs/runbooks/dns-architecture-diagnose.md, internal/diagnose/dns_architecture_report.go, and internal/diagnose/dns_architecture_test.go.

Resolve these while preserving both PRs' fixes, then test the integrated candidate and update evidence for any changed execution paths. The separate-head test results do not validate the conflict resolution.

Conflicts, resolved by keeping both sides: - cmd/pilot/cmd/mcp_diagnose_dns_architecture.go: the tier's authority probes of left-out replicas are dropped (#92), and the internal-name skip detail is kept (#91); stringInList (#92) next to #91's name helpers. - internal/diagnose/dns_architecture_report.go: a left-out replica gives only freeipa_replica_not_dns_server (#92); an unprobed provider is skipped (#91). - internal/diagnose/dns_architecture_test.go: both sides' new tests. - docs/runbooks/dns-architecture-diagnose.md: #91's status and findings rows, plus freeipa_replica_not_dns_server. Semantic conflict: #91 compares every DNS provider's answer for the internal name with the reference provider's. A replica #92 leaves out is not a FreeIPA DNS server, so its expected failure became a false mismatch. The comparison now skips left-out replicas; TestBuildDNSArchDifferences_LeftOutReplicaIsNotComparedWithTheReference fails without the change. #91's TestBuildDNSArchStatus_UnknownConfigAndComparisonsAreNamed now expects #92's subject for server forwarders, "(ipa dnsserver-mod)". The diagnose runbook gets a §6 row for #92. Co-Authored-By: Claude Opus 5.5 <[email protected]>

New record 2026-10-15-287cf79.md: the merge's conflicts and the semantic fix, run from a clean checkout of the committed candidate on a new six-VM topology, with the run metadata captured during the run (run IDs, UTC window, backing chains and base image sha256, package versions, input digests). Topology test L3/L4 failed=0, L5 20/16/7, L6 changed=0, dns.md 16 pass + 4 not applicable; the targeted checks match 5750f2a's. Diagnose is complete 14/14 with only freeipa_replica_not_dns_server for the unpromoted replica; a control binary without the left-out skip gives partial with comparison_unknown for it. Local checks from the same checkout: go test -race 57 ok, golangci-lint 0, playbook-lint, spec lint. The dns and diagnose runbooks' §0.5 point at the record and say which cases were not re-run on 287cf79; both get changelog rows. Co-Authored-By: Claude Opus 5.5 <[email protected]>

kjelly · 2026-10-15T04:41:59Z

#91 已合併進 main（306c444），依先前的約定由本 PR 解衝突。新 head 17fac7a，candidate 287cf79（tree 1cbaac4f），由 fde3ce4 與 306c444 合併而成。

合併

4 個衝突檔：MCP diagnose、report、測試、diagnose runbook，都是保留兩邊的內容。
語意衝突：fix(diagnose): name every failed or unknown DNS fact; refuse privileged principals (follow-up to #66) #91 新增的「每個 DNS provider 的內部名稱答案要跟參考 provider 一致」比對，會把本 PR 排除的 replica 也拿來比。已改成跳過被排除的 replica。
- 新測試沒有這個修正就會失敗。
- fix(diagnose): name every failed or unknown DNS fact; refuse privileged principals (follow-up to #66) #91 原本的一個測試改成預期本 PR 的 subject (ipa dnsserver-mod)。
其他 10 個執行檔：都是 main 原本的 blob，沒有改動；playbook、fixture、contract、spec 都沒有變。

驗證（從已 commit 的 287cf79 乾淨 checkout 執行，§3 metadata 都在執行當下記錄）

本機檢查：go test -race 57 個 package ok、golangci-lint 0 issues、playbook-lint 通過、spec lint 0 findings。
6 台新建 VM 的 topology test：
- L3/L4 failed=0；L5 為 20/0、16/0、7/0；L6 changed=0；dns.md 16 pass、4 N/A。
- 針對性檢查（replica named 停掉、forwarders 清空與還原、6 個 forwarder gate、安裝前的 gate）結果與 5750f2a 相同。
diagnose：
- candidate 是 complete、14/14，沒 promote 的 replica 只有 freeipa_replica_not_dns_server。
- 拿掉修正的對照 binary 對同一套 topology 是 partial，該 replica 被報成 comparison_unknown。
- stub 第一跳沒有誤報。

#91 的其他情境（tier 拒絕 client、FQDN 結尾的點、K/M／k）沒有在 287cf79 上重跑，由單元測試涵蓋；runbook §0.5 與 PR 說明都寫明了這點。

詳見 docs/evidence/freeipa-replica-dns-provider/2026-10-15-287cf79.md。

The run metadata added to the 01a64f4, 09ded7e and 5750f2a records gave the base images' sha256 computed on 2026-10-15, reasoning from the files' mtimes that they were unchanged. A digest computed after a run does not show what that run used (review of #89), so the records now name only the base image from libvirt's log and say the digest was not recorded. 287cf79's record has the digests as captured during its run. Co-Authored-By: Claude Opus 5.5 <[email protected]>

Each targeted check's finishing time comes from the run script's log, captured during the run. Co-Authored-By: Claude Opus 5.5 <[email protected]>

New record 2026-10-15-287cf79-targeted.md repeats 09ded7e's targeted checks with their 44e2d20 controls on the current candidate and a new six-VM topology, with the metadata captured during the run (backing chains and base image sha256, packages, input digests, per-step UTC, verify run IDs, diagnose audit IDs): the replica under another inventory name, a listed replica's named stopped, forwarders cleared, port form and default on both lists, a replica's own forwarders (--tags R3), twelve forwarder gate values, and both install-time gates. Results match 09ded7e's; the candidate's diagnose also names the inventory/FreeIPA name mismatch for that replica's server forwarders (declared_live_absent). 09ded7e's record is marked superseded as evidence, and the dns runbook §0.5 points at the new record. Co-Authored-By: Claude Opus 5.5 <[email protected]>

New record 2026-10-15-287cf79-round1.md repeats 01a64f4's checks on the current candidate, with main f9fb701 as the control and the metadata captured during the runs: a replica listed before it is promoted (main fails G5 and writes it as a nameserver; the candidate leaves it out with a WARNING), site.yml on all six hosts, freeipa_setup_dns: true failing closed in dns-apply and in the consumer, promotion and reruns, and diagnose before and after. Topology A needed three attempts; the first two failed outside this PR (#110: concurrent replica DNS self-register; #111: guest clocks behind after rollback) and are recorded. Topology B (one primary, the unpromoted replica) covers the consumer cases that A's three-nameserver cap hides. 01a64f4's record is marked superseded; the dns runbook §0.5 points at the new record. Co-Authored-By: Claude Opus 5.5 <[email protected]>

Co-Authored-By: Claude Opus 5.5 <[email protected]>

kjelly · 2026-10-15T11:23:52Z

依 #89 那邊 reviewer 的標準（不能用事後算的 hash 推定過去 run 的 image；所有引用為現行 evidence 的 record，都要有執行當下記錄的 §3 metadata）主動補強，新 head 2f62058，只改 evidence 與文件，candidate 仍是 287cf79。

舊 record 的推定 digest 已撤掉：我之前替 01a64f4、09ded7e、5750f2a 補的 image digest 是用「現在的 hash 加檔案 mtime」推定的，現已移除，改寫成「沒有在執行當下記錄」。三份都標為 superseded。
新增 2026-10-15-287cf79-targeted.md：在 287cf79 上重跑 round 2 的 targeted checks 與 44e2d20 對照。範圍：
- 名稱不符的 replica；
- replica named 停掉；
- forwarders 清空、port form、預設值；
- replica 自己的 forwarders（--tags R3）；
- 12 個 gate 值；
- 兩種安裝前 gate。
新增 2026-10-15-287cf79-round1.md：在 287cf79 上重跑 round 1 的情境與 main 對照。範圍：
- 還沒 promote 的 replica；
- site.yml；
- freeipa_setup_dns: true fail closed；
- promote 後重跑；
- promote 前後的 diagnose。
六節點 topology 試了三次，前兩次失敗的原因都不在本 PR，已照實記錄並開 issue：
- bug(freeipa-server-replica): 多台 replica 同時 self-register DNS 時，ipa dnsrecord-add 偶發 No valid Negotiate header #110：兩台 replica 同時 self-register DNS，ipa dnsrecord-add 偶發 No valid Negotiate header；
- bug(vm-target): rollback 到 snapshot 後，EL guest 的時鐘落後，之後的 FreeIPA 操作會失敗 #111：rollback 之後 AlmaLinux guest 的時鐘慢了約 31 分鐘。
另外用一個四節點 topology 補上 consumer 端的情境，因為 resolver 最多寫 3 個 nameserver，六節點時被排除的 replica 排不進名單。
結果都與先前一致；不同之處在 record 裡逐項寫明。
PR 說明：新增「Current evidence」一節，列出這三份 record；dns runbook §0.5 與變更紀錄也已更新。

…ca-followup

…tative BuildDNSArchPaths printed a replica the playbooks leave out (freeipa_setup_dns unset and not listed by FreeIPA as a DNS server) as "FreeIPA DNS replica … authoritative for <domain>", because it checked only DNSProvider. It now prints that the replica is left out of the DNS paths. TestBuildDNSArchPaths_LeftOutReplicaIsNotAuthoritative fails without the change. Found by the review of 2f62058. Co-Authored-By: Claude Opus 5.5 <[email protected]>

…b02b5cb The review of 2f62058 found that main had moved (#89, #101, #107) and changed files on this PR's tested path, so 287cf79's evidence did not cover the merged tree, and that the diagnose runbook still cited #91's 46fbc58 run for cases the merged code handles. New record 2026-10-15-b02b5cb.md repeats every check of the three 287cf79 records on b02b5cb (main merged + the left-out paths fix), with the 44e2d20 and main f9fb701 controls, plus #91's diagnose cases (tier refuses the client, trailing-dot FQDN, K/M and k principals, stub and FreeIPA BIND first hops), on new six- and four-VM topologies, with the metadata captured during the run. Results match; the control for the port form on 44e2d20 is added. Local checks from the same clean checkout: go test -race 57 ok, golangci-lint 0, playbook-lint, spec lint. The 287cf79 records are marked superseded; the dns and diagnose runbooks' §0.5 point at the new record. Co-Authored-By: Claude Opus 5.5 <[email protected]>

kjelly · 2026-10-15T13:55:45Z

處理對 2f62058 的獨立 review（沒有 blocker）。新 head e13daae，candidate b02b5cb（tree 38a98e94）。

main 又前進了：fix(freeipa): finish the private Kerberos cache work from #72 #89、fix(diagnose): validate DNS probe results before reporting success #101、fix(fixtures): wait for client sudo policy before simulations #107 改到本 PR 測試路徑上的檔案（private-ccache task、generator.go），所以再合併一次 main cc38f58，沒有衝突。
修 review 找到的 paths 問題：BuildDNSArchPaths 會把被排除的 replica 列成「authoritative」，現在改列「left out of the DNS paths」；新增的測試在修正前會失敗。
新 record 2026-10-15-b02b5cb.md：在 b02b5cb 上把三份 287cf79 record 的所有檢查重跑一次。
- 包含 44e2d20／main f9fb701 對照，也補上了 44e2d20 對 "1.1.1.1 port 53 9.9.9.9" 的對照（被拒）。
- 再加上 fix(diagnose): name every failed or unknown DNS fact; refuse privileged principals (follow-up to #66) #91 的情境：tier 拒絕 client、FQDN 結尾的點、K/M／k、stub 與 FreeIPA BIND 當第一跳。所以 diagnose runbook 不再引用 46fbc58 當現行證據。
- 用 6 台與 4 台兩個新 topology，從乾淨 checkout 執行，輸入都在 checkout 外，§3 metadata 都在執行當下記錄。
結果都與先前一致，各種對照也重現了舊版行為。bug(freeipa-server-replica): 多台 replica 同時 self-register DNS 時，ipa dnsrecord-add 偶發 No valid Negotiate header #110、bug(vm-target): rollback 到 snapshot 後，EL guest 的時鐘落後，之後的 FreeIPA 操作會失敗 #111 這次都沒有出現。
本機檢查（同一個乾淨 checkout）：go test -race 57 ok、golangci-lint 0、playbook-lint、spec lint 都通過。
一個附帶發現：freeipa-server-replica.md 的 C3、C12、C15 把 replica FQDN 寫死成 ipa2，對 promote 後的 ipa4 不可能通過。這不是本 PR 的問題，record 已照實說明。

287cf79 的三份 record 標為 superseded；dns 和 diagnose runbook 的 §0.5、PR 說明都已更新。

…ows, inputs - dns runbook §0.5 points at the b02b5cb record for the replica under another name; freeipa-dns-client runbook §0.5 adds the b02b5cb run; - the superseded records' banners name b02b5cb as current; - the b02b5cb record says which replica-spec rows failed in A and in B, that packages were recorded only for pr92-ipa and pr92-dns, that two inventory copies were not fingerprinted, and names main's spec files; - the diagnose runbook's §0.5 row no longer claims versions this run did not record. Co-Authored-By: Claude Opus 5.5 <[email protected]>

kjelly · 2026-10-15T14:16:13Z

依獨立 review 對 e13daae 的意見修改（沒有 blocker，五項都是引用或措辭），新 head c6bcfe9，只改文件與 evidence，candidate 仍是 b02b5cb。

dns runbook §0.5：「名稱不同的 replica」改指向 b02b5cb record 的 R2.1。
freeipa-dns-client runbook §0.5：補上 b02b5cb 的實跑（本檔 7/7，run ID e20766f0、ead2e6dc，以及 consumer 端的各情境）。
舊 record 的 superseded 說明：改成「目前的 candidate 是 b02b5cb」。
b02b5cb record：
- 更正 replica spec 那段：A 只有 C3 失敗，B 是 C3、C12、C15 失敗，並說明原因（spec 把 ipa2 寫死）；
- 寫明套件版本只記了 pr92-ipa、pr92-dns，兩份 inventory 副本沒有指紋；
- 補上從 main 帶進來的三份 spec。
diagnose runbook §0.5：不再宣稱這次沒記錄到的套件版本。
PR 說明：Round 3 的標題、「Not re-run on 287cf79」與輪數等過時文字都已更正。

CI 在 e13daae 上 4/4 通過；main 目前的 00ac4b3（#112）沒有改到本 PR 的測試路徑，合併檢查也是乾淨的。

kjelly · 2026-10-16T03:11:51Z

本次審查將 PR 轉回草稿，原因是發現以下阻擋合併的缺陷（審查 head：c6bcfe99d57cc9610e6652b0a3a5c8a04aedc5a2）。

[P2] 新增的 replica forwarder task 會展開密碼中的 shell 特殊字元

位置：playbooks/apply/freeipa-server-replica-apply.yml:730。ipa_admin_password 直接插入 shell 雙引號，密碼中的 $、反引號或 $(...) 會被 shell 解讀，導致傳給 kinit 的密碼改變，甚至執行其中的命令。no_log: true 只隱藏輸出，無法避免此問題。

具體新增的失敗路徑：對已啟用 DNS 的既有 replica 重跑 playbook，使用官方 single-host vm-target inventory（沒有 freeipa-server group）。舊版 self-register task 會跳過，enroll/promote 使用 command.argv；此次新增的 forwarder block 則會執行這個 shell kinit，使原本可處理的合法密碼失敗。

**實際重現：**原封抽出新增 task，在 localhost 用真實 ansible-playbook 執行，搭配只檢查 stdin 的 fake kinit。測試密碼為字面值 demo$pilot92_missing_suffix，且該環境變數未設定；fake kinit 實際收到 demo，回傳 rc=1，Ansible 回傳 rc=2、PLAY RECAP failed=1。這次重現沒有呼叫真實 Kerberos／FreeIPA，也未部署 VM。

**修正方向：**用 ansible.builtin.command 的 argv 呼叫 kinit，透過 stdin 傳入密碼，保留 no_log 與私有 credential cache 的清理；補上密碼含特殊字元仍原樣傳遞的 regression test。依 repo AGENTS.md §5.10 同時檢查同類寫法。

其餘 DNS／CLI focused race 測試與此 head 的 4 項 CI 均通過；通過的測試未涵蓋上述密碼情境。此缺陷修正並完成相關驗證前，先維持草稿，不合併。

…shell text The replica forwarder task added in this PR ran `printf %s "{{ ipa_admin_password }}" | kinit ...` in bash. Ansible puts the password into the command text before the shell parses it, so `$`, backticks and `$(...)` in it were expanded or run, and kinit got a different password. On an existing DNS replica with the single-host vm-target inventory this is the one kinit that runs, so a password the other steps accept made the play fail (PR #92 review). The task now runs `kinit admin@REALM` through command argv with the password on stdin; no_log and the private ccache block are unchanged. TestRegression_ReplicaForwarderKinitPassesPasswordUnchanged runs the real task with ansible-playbook and a fake kinit. On c6bcfe9's playbook, `demo$pilot92_missing_suffix` reaches kinit as `demo`, the backtick and $(...) passwords run `touch`, and a password with quotes fails the play. All six cases pass now. Searched playbooks/** for the same pattern (AGENTS §5.10): 33 tasks on main put a password-like variable into shell or free-form command text (kinit, ipa passwd, htpasswd, ldapmodify -w, kadmin.local) across apply playbooks, shared task files, check playbooks and test fixtures. TestRegression_NoSecretInCommandText now fails for any new one; the 33 are a ratchet allowlist that fails when an entry stops matching, tracked in issue #118. Co-Authored-By: Claude Opus 5.5 <[email protected]>

kjelly · 2026-10-16T03:30:26Z

針對 c6bcfe9 審查的 [P2]，修正已推到 cdcd39b，PR 先維持草稿。這次另外合併了 main 的 f98327b，與本 PR 的檔案沒有重疊。

修正

freeipa-server-replica-apply.yml 的「FreeIPA replica DNS — kinit admin (for dnsserver-mod)」改用 ansible.builtin.command：argv: [kinit, "admin@{{ ipa_realm }}"]，密碼由 stdin 傳入，不再經過 shell。no_log: true 與私有 ccache block 都保留。

Regression test

TestRegression_ReplicaForwarderKinitPassesPasswordUnchanged 從 playbook 抽出這個 task，用真實 ansible-playbook（ansible-core 2.19.2）搭配只記錄 stdin 與參數的 fake kinit 執行。測試密碼共 6 個：demo$pilot92_missing_suffix、含反引號、含 $(...)、含引號與反斜線、前後有空白、{{ … }}。
換回 c6bcfe9 的 playbook 後，有 4 個案例失敗：
- kinit 收到 demo；
- 反引號與 $(...) 中的 touch 被執行，kinit 分別收到 ab、xy；
- 含引號的密碼讓 play 失敗。
修正後 6 個案例都通過，kinit 只收到 admin@REALM 這一個參數。

同類寫法（§5.10）

掃過 playbooks/**，main 上原本就有 33 個 task 把密碼類變數放進 shell 或 free-form command 字串，涉及 kinit、ipa passwd、htpasswd、ldapmodify -w、kadmin.local，分布在 apply、共用 task 檔、check 與 test fixtures。
新增全 repo lint TestRegression_NoSecretInCommandText：新增任何一處都會失敗。
既有的 33 處列為 ratchet allowlist；某個條目不再命中時測試也會失敗，所以清單只會縮小。逐一遷移由 bug(playbooks): 33 個 task 把密碼插進 shell/command 字串，密碼中的 $、反引號、引號會被改寫或執行 #118 追蹤，因為每一處都需要對該 playbook 實跑。

本機驗證（cdcd39b）

go test ./internal/spec/ ./cmd/pilot/cmd/：通過
go vet：通過
replica playbook --syntax-check：通過
duplicate-key 檢查：通過

尚未完成：replica playbook 已變更，b02b5cb 的 evidence 對這個檔案不再有效。接下來會凍結新的 candidate，在新 VM 上重跑 b02b5cb 的完整範圍，並把 admin 密碼換成含 $、反引號、引號的值，在真實 FreeIPA 上證明密碼會原樣傳遞。完成並經獨立審查後，才會轉為 ready。

…k re-run on cdcd39b Candidate cdcd39b (tree 422ca9e) on new VMs, from a clean detached checkout, with metadata captured during the run. It repeats every check of the b02b5cb record: - topology tests A and B (L3-L6); - dns.md; - #91's diagnose cases; - round 2 (R2.1-R2.6) and round 1, with the 44e2d20 and main f9fb701 controls and the no-fix binary; - the replica spec on ipa4. All results match. It adds the review's P2 scenario: re-running the replica playbook on an existing DNS replica with the single-host vm-target inventory, after the FreeIPA admin password was changed to throwaway values containing $, backtick and $(...) commands, and quotes. - c6bcfe9 fails at the forwarder kinit. With the quote-free value, the touch commands in the password ran on the replica as root. - cdcd39b sets the forwarders, reruns with changed=0, and no command in the password ran. The record lists the git blob of all 646 execution files in 2026-10-16-cdcd39b-blobs.txt. b02b5cb's record is marked superseded, and the older banners and the dns, dns-architecture-diagnose and freeipa-dns-client runbook summaries now point to the new record. Co-Authored-By: Claude Opus 5.5 <[email protected]>

kjelly · 2026-10-16T05:34:10Z

cdcd39b 已在新 VM 上完整重跑，紀錄在 3dbc0ef 推上，只改 evidence 與文件。PR 維持草稿，等獨立審查後再決定是否轉為 ready。

Candidate：cdcd39b，tree 422ca9ec8272a5e0a709991d7a5407eba117d3ff。內容是 P2 修正，加上與 main f98327b 的合併。
紀錄：docs/evidence/freeipa-replica-dns-provider/2026-10-16-cdcd39b.md。所有 execution file 的 git blob（646 個）列在同目錄的 2026-10-16-cdcd39b-blobs.txt。

執行條件

從 candidate 的乾淨 detached checkout 執行，開始前 git status --short --ignored 為空，所有輸入都在 checkout 外。
下列 metadata 都在執行當下記錄：base image sha256、套件版本、輸入指紋、每一步的 UTC 時間、verify run ID、diagnose audit ID。
binary 用 -trimpath 建置。執行後在另一個目錄重建，sha256 相同。

重跑 b02b5cb 的完整範圍（2026-10-16 04:09–05:16 UTC，結果與 b02b5cb 一致）

Topology A（6 台）：L3–L6 通過。L5 結果：
- freeipa-server.md 20/0（1aad6b15）
- replica 16/0（cf5040a4）
- freeipa-dns-client.md 7/0（4790534e）
- dns.md 16 pass、4 n/a（674e241f）
Topology B（4 台）：L3–L6 通過。L5 結果：
- freeipa-server.md 20/0（27aee87f）
- freeipa-dns-client.md 7/0（04385fa5）
其他檢查：fix(diagnose): name every failed or unknown DNS fact; refuse privileged principals (follow-up to #66) #91 的 diagnose 情境、R2.1–R2.6、round 1，以及 B 的 consumer 情境，都與 b02b5cb 的結果一致。對照組不變，包括 44e2d20、main f9fb701 與 no-fix binary。

P2 情境實跑（依審查描述：既有的 DNS replica、單機 vm-target inventory、沒有 freeipa-server group）

先把 admin 密碼改成一次性的值，內含 $、反引號、$(touch …) 與引號，然後重跑 replica playbook：
- c6bcfe9 在 forwarder 的 kinit 失敗。
- cdcd39b 正常設定 forwarders，重跑 changed=0。
第一組密碼裡的引號讓 bash 無法解析整行，所以 $(...) 根本沒有執行。因此再用一組不含引號的密碼重做一次：
- c6bcfe9 在同一步失敗，而且密碼裡的兩個 touch 在 replica 上以 root 執行了（marker 檔在 04:56:31 出現）。
- cdcd39b 正常完成，重跑 changed=0，沒有產生任何 marker。
這項檢查只跑 replica playbook，因為 main 上其他還用 shell 的 kinit（bug(playbooks): 33 個 task 把密碼插進 shell/command 字串，密碼中的 $、反引號、引號會被改寫或執行 #118）遇到這種密碼也會失敗。

本機檢查（同一個 checkout）

go test -race：57 個 package ok
golangci-lint 2.12.1：0 issues
make playbook-lint：通過
spec lint：4 份皆 0 findings

其他

b02b5cb 的紀錄已標為 superseded，較舊紀錄的 banner 也改為指向新紀錄。
dns.md、dns-architecture-diagnose.md、freeipa-dns-client.md 三份 runbook 的摘要都已更新。
VM 已全部拆除。原始 log 與一次性 vault 會在清理時刪除，紀錄中沒有任何密碼值。

…stdin PR #92 review of 3dbc0ef: cdcd39b fixed only the forwarder kinit. With an inventory that has a freeipa-server group, re-running the replica playbook still ran the self-register kinit (`printf %s "{{ ipa_enroll_password }}" | KRB5CCNAME=... kinit`) on the primary, and the primary's own dnsconfig-mod kinit, in the block this PR changed, had the same shell printf. Both now run kinit through command argv with the password on stdin; the self-register task sets KRB5CCNAME to its dedicated cache through environment. no_log is unchanged. TestRegression_FreeIPAKinitTasksPassPasswordUnchanged (was ...ReplicaForwarderKinit...) runs all three real tasks with ansible-playbook and a fake kinit, and checks the dedicated cache for the self-register task. On 3dbc0ef's playbooks the two newly covered tasks fail 4 of 6 cases each (kinit gets `demo`, the backtick and $(...) parts run, a quote fails the play). The lint (TestRegression_NoSecretInCommandText): - counts tasks per "<path>::<task name>". A copied task under an allowlisted name failed before only if the name differed; adding a second "Kinit admin" to gateway-scope-apply.yml now fails. - matches _pass, pwd, token, api_key and psk names too, and skips names ending in _file, _path or _dir. This finds audit_demo_user_pass in freeipa-audit-demo.yml (kadmin cpw) and freeipa-audit-sim.yml (kinit). The allowlist stays at 33 sites: these two in, the two fixed tasks out (issue #118). - says to use stdin: an argv element is not rewritten, but any local user can read it in /proc/<pid>/cmdline. The test comment no longer claims vault values are not templated: they are, so a {{ in a real password is rendered before any task sees it. AGENTS.md: §4.5 point 15, a §6 line and changelog v1.53. Co-Authored-By: Claude Opus 5.5 <[email protected]>

kjelly · 2026-10-16T06:58:12Z

這次回覆 3dbc0ef 的獨立審查意見。程式修正已推到 dee1fe8，PR 仍維持草稿。

B1：同一支 playbook 還有兩處 shell kinit

cdcd39b 只修了 forwarder 的 kinit。只要 inventory 有 freeipa-server group，重跑 replica playbook 時仍會執行 self-register 的 printf … | kinit；而本 PR 改過的 primary block 裡，dnsconfig-mod 的 kinit 也是同樣的寫法。

現在這兩處都改成 argv 加 stdin。self-register 的 KRB5CCNAME 改由 task 的 environment 指向專用 cache。

行為測試改名為 TestRegression_FreeIPAKinitTasksPassPasswordUnchanged，用表格涵蓋這 3 個真實 task，並檢查 self-register 確實使用專用 cache。換回 3dbc0ef 的 playbook 時，新加入的兩個 task 各有 4/6 個案例失敗。

Lint

S2：allowlist 改成記錄每個 <檔案>::<task 名稱> 有幾個 task。之前在已列入的檔案裡複製一個同名 task，lint 照樣會過。現在在 gateway-scope-apply.yml 多加一個「Kinit admin」，lint 會失敗。（commit message 裡這句的寫法有誤，以這裡為準。）
S3：比對的名稱增加 _pass、pwd、token、api_key、psk，並排除 _file、_path、_dir 結尾的名稱。因此抓到 audit_demo_user_pass 的兩處（audit-demo 的 kadmin cpw、audit-sim 的 kinit）。allowlist 仍是 33 處：移除修好的 2 處，加入新抓到的 2 處。bug(playbooks): 33 個 task 把密碼插進 shell/command 字串，密碼中的 $、反引號、引號會被改寫或執行 #118 已同步更新。
S4：更正測試註解。vault 與 -e 的值也會被 Ansible 模板化，所以實際密碼裡的 {{ 在任何 task 拿到之前就已經被 render 了。這是 repo 原本就有的限制，不屬於這些 task 的範圍。
N1：lint 訊息、AGENTS.md 與 bug(playbooks): 33 個 task 把密碼插進 shell/command 字串，密碼中的 $、反引號、引號會被改寫或執行 #118 都改成「用 stdin」。argv 雖然不會被改寫，但 /proc/<pid>/cmdline 讀得到，只適合沒有其他輸入方式的程式（FreeIPA installer）。
N4：AGENTS.md 加上 §4.5 第 15 點、§6 一條規則，以及 changelog v1.53。fix(playbooks): log-server and audit-log-forwarding read the inputs they share #79 的 v1.50、feat(login-probe): add external SSH/PAM probe and host I/O alerts #80 的 v1.52 由後合併者重新編號。

本機驗證（dee1fe8）

go test ./internal/spec/ ./cmd/pilot/cmd/、go vet、兩支 playbook 的 --syntax-check、duplicate-key 檢查都通過。
既有的 kinit 私有 cache lint 全部通過。

接下來

primary 與 replica 的 playbook 都改了，要凍結新的 candidate，再整輪重跑 VM。這次用 topology（多主機）inventory 加上含特殊字元的密碼，重跑 replica playbook，也就是 B1 描述的那條路徑。

文件類的意見會在新 record 一起處理：S1 的 dns.md 單機條件、N2 把 contracts/ 列入 blob 清單、N3 時態、N5 PR 內文的過時段落、N6 說明 CI 的 Go test job 沒有 ansible。

#106 now carries #103 merged with main, so the identity playbook's reconcile tasks sit inside main's private Kerberos credential cache block. This branch's playbook change is re-applied inside that block: the decommission HBAC and sudo reference queries list every rule with `--all --sizelimit=0` and fail on an error or a truncated result. Ignoring indentation, the change is the same as on candidate 0cad952. Version numbers: - docs/verification/freeipa-identity.md: main took v1.12, #103 v1.13 and #106 v1.14, so this branch's entry is v1.15. - AGENTS.md: main has v1.49 and v1.51, and its v1.51 row notes that #79 also adds v1.50. Open PRs use v1.50 (#79, #92), v1.52 (#80, #92) and v1.53 (#92), so §4.5 point 15's changelog entry is v1.54. Point 15 follows main's rewritten point 14. The tests needed no change: the option lint and the rule-reference tests reach the tasks inside the block. Putting `--host=` back fails TestRegression_PlaybookIPAOptionsExistInIPAHelp; dropping the sudo truncation check fails TestRegression_IdentityRuleReferenceQueriesStopOnErrors. Co-Authored-By: Claude Opus 5.5 <[email protected]>

b82a0c9 merges main afeacbd into dee1fe8: the replica self-register and primary dnsconfig-mod kinits now take the password on stdin too. It was run on new VMs from a clean detached checkout, with metadata captured during the run. It repeats every check of the cdcd39b record: - topology tests A and B (L3-L6) and dns.md; - #91's diagnose cases; - round 2 (R2.1-R2.6) and round 1, with the 44e2d20 and main f9fb701 controls and the no-fix binary; - the replica spec on ipa4. All results match. All three changed kinits ran in L4 and L6 with the vault password. Special-character admin passwords (one value with quotes, one without), each control against the candidate: - (a) single-host inventory, replica playbook: c6bcfe9 fails at the forwarder kinit, and the touch commands in the password run on the replica; - (b) topology inventory, where self-register runs delegated to the primary: 3dbc0ef fails at the self-register kinit, and the commands run on the primary; - (c) freeipa-server-apply.yml --tags freeipa-verify, whose only password-reading command task is the dnsconfig-mod kinit: 3dbc0ef fails there, and the commands run on the primary. The candidate succeeds and reruns with changed=0 in every case, with no markers. The password was restored afterwards; the log scan found none. Review fixes: - S1: the dns runbook result row states the inventory condition for each case and that #118's shell kinits still break. - N2: the blob list adds contracts/ and ansible_callback/ and says which directories count. - N3: the record's tenses match what has happened. - N6: the behavioural test's evidence is the local -v run (18/18 on ansible-core 2.19.2). CI's runner image lists Ansible 2.21.5, so the test may run there rather than skip. cdcd39b's record is marked superseded. Its banner notes that its diagnose server used an empty --dir, so contracts were not loaded. The older banners point to the new record. Co-Authored-By: Claude Opus 5.5 <[email protected]>

kjelly · 2026-10-16T10:52:36Z

第 6 輪實跑完成，evidence 已推到 bf3cf42，只改 evidence 與文件。PR 仍維持草稿，等獨立審查後再決定是否轉為 ready。

Candidate

b82a0c9，tree b733b150723485349bf915a9f15084ca9cc37798：dee1fe8 合併 main afeacbd 而成。afeacbd 帶入的是 724cc45 對 internal/outbound 的 webhook 修正，加上一份 evidence，與本 PR 的路徑沒有重疊。
CI 在 b82a0c9 上 4/4 通過（run 38880469725）。
紀錄：docs/evidence/freeipa-replica-dns-provider/2026-10-16-b82a0c9.md。688 個 execution file 的 blob 列在 2026-10-16-b82a0c9-blobs.txt，這次加入了 contracts/ 與 ansible_callback/，並寫明哪些目錄算 execution file（N2）。
從乾淨的 detached checkout 執行，所有輸入都在 checkout 外，metadata 都在執行當下記錄。binary 用 -trimpath 建置，在其他目錄重建後 hash 相同。

完整重跑 cdcd39b 的範圍（2026-10-16 09:28–10:34 UTC，結果都與前一輪一致）

Topology A（6 台）：L3–L6 通過。L5 結果：
- freeipa-server.md 20/0（7013a951）
- replica 16/0（c00792a7）
- freeipa-dns-client.md 7/0（023cd719）
- dns.md 16 pass、4 n/a（523d2080）
Topology B（4 台）：L3–L6 通過。L5 結果：
- freeipa-server.md 20/0（4dfbc1fd）
- freeipa-dns-client.md 7/0（d3ace139）
3 個改過的 kinit：在 L4 與 L6 都以 vault 密碼 ok。self-register 的 kinit 委派到 primary 執行。
其他：fix(diagnose): name every failed or unknown DNS fact; refuse privileged principals (follow-up to #66) #91 的 diagnose 情境、R2.1–R2.6、round 1（含 site.yml 與 promote）、B 的 consumer 情境都已重跑，對照組不變。
diagnose 的 --dir：這輪改用各自 binary 的 checkout，所以 supported layer 讀得到 contracts/。cdcd39b 那輪用的是空目錄，已在它的 banner 註明。

特殊字元密碼實跑（兩組一次性密碼：一組含引號，一組不含）
對照組都失敗；用不含引號的那組時，密碼裡的 touch 以 root 執行。b82a0c9 三種情境都正常完成、重跑 changed=0，沒有產生 marker。

(a) 單機 inventory（沒有 freeipa-server group），replica playbook：c6bcfe9 在 forwarder 的 kinit 失敗，指令在 replica 上執行。
(b) topology inventory（self-register 委派到 primary），replica playbook：3dbc0ef 在 self-register 的 kinit 失敗，指令在 primary 上執行。
(c) primary 的 freeipa-server-apply.yml --tags freeipa-verify：會讀 admin／DM 密碼的 command task 共 7 個，這個 tag 只選到 dnsconfig-mod 的 kinit。3dbc0ef 在這個 kinit 失敗，指令在 primary 上執行。
沒有用特殊密碼跑的部分：389-ds audit 那支 ldapmodify -w（tag freeipa-audit，bug(playbooks): 33 個 task 把密碼插進 shell/command 字串，密碼中的 $、反引號、引號會被改寫或執行 #118）不在這個 tag 內，這次沒有用特殊密碼跑。bug(playbooks): 33 個 task 把密碼插進 shell/command 字串，密碼中的 $、反引號、引號會被改寫或執行 #118 其他還在用 shell 的 kinit 遇到這種密碼仍會失敗，所以完整的 apply 鏈也沒有用特殊密碼跑。
跑完後 admin 密碼已改回 vault 的值，並用它實際跑過一次。掃描全部 284 份 log 與 diagnose 結果，沒有找到任何密碼。

N6 需要更正
CI 的 Go test job 雖然沒有安裝 Ansible，但 runner image（ubuntu-24.04 20261012.334）本身內含 Ansible 2.21.5。所以這個測試在 CI 上可能會實際執行，而不是被 skip；go test 沒加 -v，看不出是哪一種。紀錄只把本機的 -v 結果當作證據：ansible-core 2.19.2，18/18 subtest 通過。

其他文件修正

S1：dns.md §0.5 的結果欄寫明三種情境各自的 inventory 條件，並註明 bug(playbooks): 33 個 task 把密碼插進 shell/command 字串，密碼中的 $、反引號、引號會被改寫或執行 #118 的其他 shell kinit 仍會失敗。
N3：時態依實際狀況：VM 已拆除；原始檔會在 commit 之後刪除。
N5：已移除 PR 內文裡過時的「round 4」那句。
superseded：cdcd39b 的紀錄已標為 superseded，較舊的 banner 都改為指向新紀錄。
runbook：三份 runbook 的摘要都已更新。

本機檢查（同一個 checkout）

go test -race：57 個 package ok
golangci-lint 2.12.1：0 issues
make playbook-lint：通過
spec lint：4 份都是 0 findings

kjelly and others added 2 commits October 14, 2026 18:33

kjelly mentioned this pull request Oct 14, 2026

fix(diagnose): name every failed or unknown DNS fact; refuse privileged principals (follow-up to #66) #91

Merged

kjelly and others added 2 commits October 15, 2026 01:57

docs(evidence): PR #92 second review verified on 09ded7e

e4935aa

Co-Authored-By: Claude Opus 5.5 <[email protected]>

kjelly and others added 4 commits October 15, 2026 04:26

docs(evidence): state the replica forwarder runs as they ran

af55ed2

Co-Authored-By: Claude Opus 5.5 <[email protected]>

docs(evidence): PR #92 review fixes verified on 5750f2a

26e5cfb

Co-Authored-By: Claude Opus 5.5 <[email protected]>

docs(runbooks): dns §0.5 says the 5750f2a VMs were newly created

a2f8e68

The 5750f2a run created a new set of six VMs with `topology up`; the §0.5 line still described the 09ded7e run's `topology reset`. Docs only. Co-Authored-By: Claude Opus 5.5 <[email protected]>

kjelly mentioned this pull request Oct 14, 2026

fix(freeipa-dns-client): nameserver check 把 tier 的 SERVFAIL 當成「不服務 zone」並給出錯誤的修復建議 #98

Closed

kjelly marked this pull request as ready for review October 14, 2026 22:02

kjelly commented Oct 15, 2026

View reviewed changes

kjelly marked this pull request as draft October 15, 2026 00:00

kjelly commented Oct 15, 2026

View reviewed changes

kjelly and others added 2 commits October 15, 2026 12:08

kjelly and others added 3 commits October 15, 2026 14:35

docs(evidence): 287cf79 record gives each targeted check its UTC time

f41ff74

Each targeted check's finishing time comes from the run script's log, captured during the run. Co-Authored-By: Claude Opus 5.5 <[email protected]>

This was referenced Oct 15, 2026

bug(freeipa-server-replica): 多台 replica 同時 self-register DNS 時，ipa dnsrecord-add 偶發 No valid Negotiate header #110

Open

bug(vm-target): rollback 到 snapshot 後，EL guest 的時鐘落後，之後的 FreeIPA 操作會失敗 #111

Closed

kjelly and others added 2 commits October 15, 2026 19:23

docs(evidence): mark the 5750f2a record superseded by 287cf79's

2f62058

Co-Authored-By: Claude Opus 5.5 <[email protected]>

Merge remote-tracking branch 'origin/main' into fix/freeipa-dns-repli…

d878d2e

…ca-followup

kjelly and others added 2 commits October 15, 2026 20:29

kjelly marked this pull request as ready for review October 15, 2026 14:53

kjelly mentioned this pull request Oct 16, 2026

ci: raise the Go test timeout to 30m #116

Merged

3 tasks done

kjelly marked this pull request as draft October 16, 2026 02:14

Merge origin/main into fix/freeipa-dns-replica-followup

d47c2e0

kjelly mentioned this pull request Oct 16, 2026

bug(playbooks): 33 個 task 把密碼插進 shell/command 字串，密碼中的 $、反引號、引號會被改寫或執行 #118

Open

Merge origin/main (afeacbd) into fix/freeipa-dns-replica-followup

b82a0c9

kjelly mentioned this pull request Oct 16, 2026

fix(decommission): find the rules naming a host and ignore roster tombstones #109

Merged

kjelly marked this pull request as ready for review October 19, 2026 01:16

kjelly merged commit 035c627 into main Oct 19, 2026
4 checks passed

kjelly deleted the fix/freeipa-dns-replica-followup branch October 19, 2026 01:24

kjelly mentioned this pull request Oct 30, 2026

feat(skills): dispute wrong claims, align PR and evidence gates, refresh VM and roster guidance #201

Merged

Conversation

kjelly commented Oct 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Breaking change

Current evidence (candidate b82a0c9)

Round 6: the self-register and dnsconfig-mod kinits (review of 3dbc0ef)

Round 5: the replica forwarder kinit (review of c6bcfe9; candidate cdcd39b, superseded by round 6)

Round 4: merged with main again; left-out replicas in the paths (candidate b02b5cb, superseded by round 5)

Round 2

Round 1 (01a64f4, unchanged)

Verification (round 2)

Round 3: merged with main (#91), candidate 287cf79 (superseded by round 4)

Uh oh!

kjelly commented Oct 14, 2026

Uh oh!

kjelly commented Oct 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kjelly left a comment

Choose a reason for hiding this comment

Uh oh!

kjelly commented Oct 15, 2026

Uh oh!

kjelly left a comment

Choose a reason for hiding this comment

Uh oh!

kjelly commented Oct 15, 2026

Uh oh!

kjelly commented Oct 15, 2026

Uh oh!

kjelly commented Oct 15, 2026

Uh oh!

kjelly commented Oct 15, 2026

Uh oh!

kjelly commented Oct 16, 2026

Uh oh!

kjelly commented Oct 16, 2026

Uh oh!

kjelly commented Oct 16, 2026

Uh oh!

kjelly commented Oct 16, 2026

Uh oh!

kjelly commented Oct 16, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

kjelly commented Oct 14, 2026 •

edited

Loading

Current evidence (candidate `b82a0c9`)

Round 6: the self-register and dnsconfig-mod `kinit`s (review of `3dbc0ef`)

Round 5: the replica forwarder `kinit` (review of `c6bcfe9`; candidate `cdcd39b`, superseded by round 6)

Round 4: merged with `main` again; left-out replicas in the paths (candidate `b02b5cb`, superseded by round 5)

Round 1 (`01a64f4`, unchanged)

Round 3: merged with `main` (#91), candidate `287cf79` (superseded by round 4)

kjelly commented Oct 14, 2026 •

edited

Loading