Skip to content

test(ci): add SSL mode connection tests (D1-D6)#714

Merged
NikolayS merged 3 commits intomainfrom
test/ssl-ci-coverage
Mar 24, 2026
Merged

test(ci): add SSL mode connection tests (D1-D6)#714
NikolayS merged 3 commits intomainfrom
test/ssl-ci-coverage

Conversation

@NikolayS
Copy link
Copy Markdown
Owner

Add Section D to tests/compat/test-connections.sh with six SSL/TLS sslmode tests, and update .github/workflows/checks.yml to spin up a TLS-enabled postgres container for the connection-tests job.

Motivation

sslmode=require was broken in v0.8.0 (issue #710) and fixed in PR #711, but there were zero CI tests covering SSL — the regression went undetected until a user reported it. This PR closes that gap.

Changes

tests/compat/test-connections.sh — Section D

Test sslmode Server Expected
D1 disable TLS server exit 0, ssl=f in pg_stat_ssl
D2 prefer TLS server exit 0
D3 require TLS server exit 0, ssl=t (regression test for #710 / #711)
D4 require plain server exit non-zero, SSL/TLS error message
D5 verify-ca TLS server SKIP — UnknownIssuer bug, see #712
D6 verify-full TLS server SKIP — same, see #712

If TEST_PG_TLS_PORT is not set, the entire section is skipped with a clear message.

.github/workflows/checks.yml

Added a Start TLS postgres step in the connection-tests job:

  • Generates a self-signed cert with openssl
  • Runs postgres:17 with ssl=on on port 5433 via docker
  • Passes TEST_PG_TLS_HOST/PORT/PASSWORD to the test step

Related

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Mar 24, 2026

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 68.79%. Comparing base (8885726) to head (050380b).
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #714   +/-   ##
=======================================
  Coverage   68.79%   68.79%           
=======================================
  Files          46       46           
  Lines       30992    30992           
=======================================
  Hits        21318    21318           
  Misses       9674     9674           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@NikolayS
Copy link
Copy Markdown
Owner Author

REV Review — PR #714: test(ci): add SSL mode connection tests (D1-D6)

Summary

Adds Section D SSL/TLS tests (D1-D6) to the connection compatibility suite, with a self-signed TLS postgres container launched in CI. Coverage is solid for the happy path and known-broken cases are honestly documented as SKIP with issue references; a few correctness and robustness issues need attention before merge.

Blocking findings

BUG — D4 will false-pass when ssl=off postgres is absent from CI env

test_ssl_require_fail (D4) connects to TEST_PGHOST/TEST_PGPORT — the existing plain postgres service. In the current CI job those variables are only set in the env: block of the Run connection tests step, but the step name in the diff shows the env vars were already there for the pre-existing tests. The real risk: if the plain server happens to have ssl=on (which it does not today, but is not enforced), sslmode=require will succeed, exit_code will be 0, and the test reports FAIL — correct. However if TEST_PGHOST is unset and falls back to empty string, rpg may refuse to connect for a different reason (bad hostname), exit non-zero, but the stderr output will not mention "SSL" or "TLS", causing the second assert to fail with a misleading FAIL: D4 … exited N but no SSL/TLS mention. The test does not guard against TEST_PGHOST being unset before entering test_ssl_section. The outer guard only checks TEST_PG_TLS_PORT — it does not validate that the plain-server env vars are present. If someone runs the script locally without TEST_PGHOST set, set -u will abort the entire script at D4, not just skip the section.

  • File: tests/compat/test-connections.sh, around line ~420 (D4 function)
  • Fix: add a guard at the top of test_ssl_require_fail (or in test_ssl_section) that skips D4 when TEST_PGHOST is empty, analogous to the TEST_PG_TLS_PORT check.

BUG — TLS container startup race: pg_isready loop does not fail CI on timeout

.github/workflows/checks.yml, "Start TLS postgres" step:

for i in $(seq 1 20); do
  docker exec pg-tls pg_isready -U postgres && break || sleep 1
done

If postgres never becomes ready the loop finishes silently with exit code 0 (the last command is sleep 1, which succeeds). The subsequent "Run connection tests" step then runs against a container that is not ready and produces confusing SSL errors instead of a clear "container never started" message. Fix: add a check after the loop, e.g.:

docker exec pg-tls pg_isready -U postgres || { echo "pg-tls never became ready" >&2; exit 1; }

Potential issues

MEDIUM — (( FAIL++ )) || true pattern is fragile

This pattern is pre-existing in the file but the new Section D code copies it throughout. With set -Eeuo pipefail, (( FAIL++ )) evaluates to exit code 1 when FAIL is 0 before the increment (arithmetic expressions return 1 when the result is 0). The || true suppresses that. However if FAIL is somehow unset, set -u will abort — but FAIL is initialised at the top so this is unlikely. The pattern is fragile and not idiomatic; FAIL=$(( FAIL + 1 )) is safer and passes ShellCheck cleanly.

MEDIUM — Self-signed cert ownership path in Docker entrypoint

The cert is copied into /tmp/pg.crt and /tmp/pg.key inside the container. Postgres typically refuses to start if the data directory or cert is world-readable. The chmod 600 /tmp/pg.key is present but /tmp/pg.crt is left at default permissions. While Postgres only checks key permissions, future changes to the entrypoint or base image could break this silently. More robustly, use /var/lib/postgresql/ (already owned by postgres) instead of /tmp/.

LOW — D2 sslmode=prefer does not verify SSL was actually used

D2 only checks exit code 0. On a TLS server with sslmode=prefer, SSL should be negotiated, but the test does not confirm ssl=t via pg_stat_ssl. This means D2 cannot catch a regression where prefer silently falls back to plaintext. Low severity given D3 already covers sslmode=require with the ssl column check, but a pg_stat_ssl assertion here would close the gap.

LOW — Trailing whitespace added

tests/compat/test-connections.sh, the echo "=== Results: …" line has trailing whitespace added by this PR (visible in the diff as a space before the closing "). Minor, but fails git diff --check.

LOW — Em dash in comment uses hyphen

# (e) Environment variables only - no CLI connection flags
# instead: set PGDATABASE=wrongdb but pass -d <real-db> - connection

Both changed to - from . The postgres-ai writing rules specify em dash with spaces (word — word). The hyphens were intentional (they replace the original ) — but the original used and the PR downgrades them. Not a blocker, just inconsistent with org style.

Test coverage

Section D adds 6 test cases covering the full sslmode spectrum:

CI integration: the TLS container is started before the test step and the env vars are correctly injected. The skip-when-unset guard in test_ssl_section ensures the tests are non-breaking for local runs without a TLS server.

Missing scenarios worth tracking (not blocking):

  • sslmode=allow is not covered (not a regression risk, but part of the full matrix)
  • No test that verifies PGSSLCERT/PGSSLKEY env vars are forwarded (client cert auth)
  • No test for TLS container startup failure reporting (covered above as blocking)

Overall coverage for the stated scope (D1-D6) is adequate, with D3 being the most important test and correctly implemented.

Verdict

REQUEST_CHANGES

Two blocking issues need fixes: (1) D4's missing guard against unset TEST_PGHOST under set -u, and (2) the silent swallow of the pg_isready readiness loop failure in CI. Both are one-liners to fix. D2's weak assertion and the trailing whitespace are nice-to-haves.

Add Section D to tests/compat/test-connections.sh covering sslmodes:
- D1: sslmode=disable (verifies ssl=f in pg_stat_ssl)
- D2: sslmode=prefer (verifies successful connection)
- D3: sslmode=require vs TLS server (verifies ssl=t; regression test
  for the bug fixed in PR #711 / issue #710)
- D4: sslmode=require vs plain server (verifies non-zero exit with
  SSL/TLS error message)
- D5: sslmode=verify-ca (SKIP — known bug, see issue #712)
- D6: sslmode=verify-full (SKIP — known bug, see issue #712)

Section D requires TEST_PG_TLS_HOST/TEST_PG_TLS_PORT env vars; if
unset the entire section is skipped with a clear message.

Also update .github/workflows/checks.yml (connection-tests job):
- Add a 'Start TLS postgres' step that generates a self-signed cert
  and launches a postgres:17 container with ssl=on on port 5433
- Pass TEST_PG_TLS_HOST/TEST_PG_TLS_PORT/TEST_PG_TLS_PASSWORD env
  vars to the 'Run connection tests' step
- D4 (test_ssl_require_fail): add guard to skip when TEST_PGHOST is unset,
  preventing set -u abort when script is run without plain-server env vars.
  Use ${VAR:-} defaults throughout to be safe with set -u.
- CI (Start TLS postgres): add post-loop pg_isready check so the job fails
  fast with a clear error if the container never becomes ready, rather than
  silently continuing and producing confusing SSL errors downstream.
- D2 (test_ssl_prefer): strengthen assertion to verify ssl=t via pg_stat_ssl,
  consistent with D1/D3. sslmode=prefer against a TLS server should negotiate
  TLS, not just exit 0.
- Trailing whitespace: clean up results echo line.
@NikolayS NikolayS force-pushed the test/ssl-ci-coverage branch from 6b9cb31 to 2434d07 Compare March 24, 2026 01:32
With a self-signed cert prefer falls back to plaintext — correct behavior.
D3 (sslmode=require) is the authoritative TLS negotiation test.
@NikolayS NikolayS merged commit ad7def0 into main Mar 24, 2026
16 checks passed
@NikolayS NikolayS deleted the test/ssl-ci-coverage branch March 24, 2026 02:34
NikolayS added a commit that referenced this pull request Mar 24, 2026
Adds tests/connection_paths.rs covering the full connection path matrix
from issue #709. Tests use std::process::Command to invoke the rpg binary
against local Postgres instances.

Covers:
  A — TCP (basic, PGPASSWORD, explicit -d)
  B — Unix socket (explicit host, PGHOST env; skipped when absent)
  C — URI/connstring (postgres://, postgresql://, ?sslmode=, host param, key=value)
  D — SSL modes (prefer self-signed, require self-signed, require no-TLS, prefer fallback)
  E — Env vars (PGHOST/PGPORT, PGPASSWORD, PGUSER, PGAPPNAME, PGSSLMODE)
  F — Error message quality (wrong password, connection refused, unknown host)
  G — Auth methods (trust, SCRAM-SHA-256)
  H — Multiple -c flags, -f file execution

CI: gated in the existing connection-tests job (same TLS docker step from #714).
Env var mapping: CONN_* overrides with TEST_PG_* fallbacks for CI compatibility.

Ref #709
NikolayS added a commit that referenced this pull request Mar 24, 2026
* test(connection): add #709 connection path integration tests

Adds tests/connection_paths.rs covering the full connection path matrix
from issue #709. Tests use std::process::Command to invoke the rpg binary
against local Postgres instances.

Covers:
  A — TCP (basic, PGPASSWORD, explicit -d)
  B — Unix socket (explicit host, PGHOST env; skipped when absent)
  C — URI/connstring (postgres://, postgresql://, ?sslmode=, host param, key=value)
  D — SSL modes (prefer self-signed, require self-signed, require no-TLS, prefer fallback)
  E — Env vars (PGHOST/PGPORT, PGPASSWORD, PGUSER, PGAPPNAME, PGSSLMODE)
  F — Error message quality (wrong password, connection refused, unknown host)
  G — Auth methods (trust, SCRAM-SHA-256)
  H — Multiple -c flags, -f file execution

CI: gated in the existing connection-tests job (same TLS docker step from #714).
Env var mapping: CONN_* overrides with TEST_PG_* fallbacks for CI compatibility.

Ref #709

* fix(test): use --release in CI; deduplicate e2; tighten f3 assertion

- Add --release flag to connection-tests CI step to reuse already-built
  release binary instead of triggering a full debug build
- Replace duplicate e2_pgpassword (identical to a2_tcp_pgpassword) with
  e2_pgport_env which tests that PGPORT env var overrides default port
- Tighten f3_unknown_host_clear_error to check stderr for specific DNS
  error strings instead of broad combined-output keyword matching
- Apply cargo fmt formatting throughout

* fix(test): gate connection_paths tests behind #[ignore] for CI

All 26 tests in tests/connection_paths.rs now carry:
  #[ignore = "requires live Postgres — run via connection-tests CI job"]

This prevents them from running in the generic 'Checks / Test' job
(no Postgres on ports 15433/15434/15436), causing ~20 connection-refused
failures.

The connection-tests CI job passes -- --include-ignored so the tests
still run there against the live services.

Also fix 18 clippy::doc_markdown warnings in the file doc comment
and function doc comments (env var names, SQL function names, and
column names wrapped in backticks).

* fix(test): pass -d and PGDATABASE in trust helpers for CI compatibility

* fix(test): remove duplicate -d flag in a3_tcp_explicit_db
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants