feat(webhooks): add json-post default preset so url-only webhooks work (#676)#677
Conversation
docs/webhooks.md "Custom Webhooks" section promised that a webhook with
just `url = ...` set would POST a JSON payload to that URL — no custom
preset YAML required. The code rejected this: `loader.Load("")` returned
an error before any send code ran, and there was no bundled preset that
turned a bare URL into a working webhook. The doc snippet has been false
since the webhook system shipped.
Fixes #676 by:
- Adding a bundled `json-post` preset (middlewares/presets/json-post.yaml,
embedded into the binary). url_scheme `{url}`, method POST,
Content-Type application/json, body shaped as
{job:{...}, execution:{...}, host:{...}, ofelia:{...}}. Stdout / stderr
are truncated to 4000 chars to keep payloads modest. The optional
`link` field is included when set.
- Adding `webhook-default-preset` global, *string-typed so unset-vs-set
is unambiguous (same pattern as SaveConfig.RestoreHistory):
nil → bundled DefaultPresetName ("json-post")
non-nil "" → explicit opt-out; webhooks missing `preset` fail
non-nil "X" → operator's chosen fallback preset name
Resolution happens at webhook attach time via
WebhookGlobalConfig.EffectiveDefaultPreset(), so live INI reloads and
label changes take effect on the next attach without restart.
- Wiring the fallback into NewWebhook: when config.Preset == "", fill
from loader.DefaultPreset() before Validate runs.
- Wiring the global through every existing seam: INI parse (gcfg),
mapstructure, docker label allow-list (operator-tunable; not
SSRF-sensitive — narrowing or widening the default cannot reach a new
destination because the URL allow-list and preset loader still gate
delivery), mergeWebhookGlobals (nil-check instead of sentinel-compare),
applyGlobalWebhookLabels.
- Rewriting docs/webhooks.md "Custom Webhooks" to describe what now
actually works, including the payload shape and the opt-out path.
Tests added:
- TestBundledPreset_JSONPostExists: preset embeds and validates.
- TestEffectiveDefaultPreset_ThreeIntents: nil / empty / explicit, plus
nil-receiver safety.
- TestNewWebhook_URLOnly_UsesJSONPostFallback: end-to-end — url-only
config attaches and the test server receives a valid JSON POST.
- TestNewWebhook_DefaultPresetOptOut: setting DefaultPreset=&"" restores
the pre-fallback "either preset or url" error path.
Signed-off-by: Sebastian Mendel <[email protected]>
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
There was a problem hiding this comment.
Automated approval for maintainer PR
All automated quality gates passed. See SECURITY_CONTROLS.md for compensating controls.
✅ Mutation Testing ResultsMutation Score: 95.45% (threshold: 60%)
What is mutation testing?Mutation testing measures test quality by introducing small changes (mutations) to the code and checking if tests detect them. A higher score means better test effectiveness.
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #677 +/- ##
==========================================
+ Coverage 87.98% 87.99% +0.01%
==========================================
Files 89 89
Lines 11216 11280 +64
==========================================
+ Hits 9868 9926 +58
- Misses 1099 1104 +5
- Partials 249 250 +1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
This PR implements the previously documented “URL-only webhook” behavior by introducing a bundled json-post preset and a new global selector ([global] webhook-default-preset) used as the fallback when a webhook omits preset.
Changes:
- Add bundled
json-postwebhook preset and make it the default fallback for URL-only webhooks. - Add
WebhookGlobalConfig.DefaultPreset(*string) with three-state semantics and wire it through preset loading and label merge paths. - Expand docs, tests, and changelog to cover the new default behavior and opt-out mechanism.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
middlewares/webhook.go |
Applies loader-provided default preset when per-webhook preset is omitted. |
middlewares/webhook_test.go |
Adds tests for bundled preset existence, three-intent default semantics, URL-only attach/dispatch, and opt-out. |
middlewares/webhook_config.go |
Introduces DefaultPresetName and EffectiveDefaultPreset() plus DefaultPreset *string in global config. |
middlewares/presets/json-post.yaml |
Adds the bundled JSON POST preset used as the default fallback. |
middlewares/preset.go |
Adds (*PresetLoader).DefaultPreset() helper to surface the effective global fallback preset. |
docs/webhooks.md |
Documents webhook-default-preset, URL-only webhook usage, payload shape, and opt-out. |
cli/docker-labels.go |
Allow-lists webhook-default-preset for Docker label ingestion. |
cli/config_webhook.go |
Adds the new global key constant, merges the new global, and applies the label into live config. |
CHANGELOG.md |
Adds an unreleased entry describing the new default preset and URL-only webhook behavior. |
Five review comments + codecov/patch failure (61% on diff, target 80%).
Code/comment fixes:
- middlewares/webhook.go: NewWebhook now returns an explicit error when
loader is nil instead of dereferencing nil later. The "loader may be
nil in tests" comment was wrong — the function unconditionally calls
loader.Load() further down.
- cli/config_webhook.go: stale doc comment said the default was
"generic"; now references middlewares.DefaultPresetName ("json-post")
consistently with the rest of the code and the docs.
Doc fixes (docs/webhooks.md):
- INI-vs-labels note rewritten: webhook-default-preset is now also
label-accepted; webhook-preset-cache-ttl already had label support
(the "not yet implemented" claim was stale since #640).
- Content-Type aligned with the preset YAML: application/json; charset=utf-8.
- Opt-out wording clarified: with the fallback disabled, every webhook
must declare preset explicitly — `url = ...` alone does NOT help,
since `url` only overrides the preset's url_scheme.
Coverage (codecov/patch fail → fix):
- cli/TestApplyGlobalWebhookLabels_DefaultPreset: 4 sub-tests covering
missing label / empty (opt-out) / custom value / non-string ignored.
- cli/TestMergeWebhookGlobals_DefaultPreset: 5 sub-tests covering all
combinations of nil-vs-set on dst/src, plus explicit-INI-wins-over-label.
- middlewares/TestNewWebhook_DefaultPresetCustom: operator's-choice arm
of EffectiveDefaultPreset (was untested in isolation from the unit
test on the accessor).
- middlewares/TestNewWebhook_ExplicitPresetWinsOverDefault: per-webhook
`preset = X` overrides any global default — the fallback is only a
fallback.
- middlewares/TestNewWebhook_NilLoaderReturnsError: pins the new
nil-guard contract added in this commit.
- middlewares/TestJSONPostPreset_FailedExecutionRendersValidJSON:
end-to-end on the FAILURE path with JSON-hostile chars (quotes,
newlines, backslashes) in job name, command, and error. Asserts the
body parses as JSON and the escaped fields round-trip cleanly. This
is the real regression value — any future refactor of the json-post
template that forgets to wrap a string field through the `json`
helper will fail here.
- middlewares/TestJSONPostPreset_OutputTruncation: pins the documented
4000-char truncation of stdout/stderr.
mergeWebhookGlobals + applyGlobalWebhookLabels + EffectiveDefaultPreset
+ PresetLoader.DefaultPreset are now all at 100% statement coverage.
Signed-off-by: Sebastian Mendel <[email protected]>
There was a problem hiding this comment.
Automated approval for maintainer PR
All automated quality gates passed. See SECURITY_CONTROLS.md for compensating controls.
…e, DX Six parallel reviewers (security, anti-pattern, continuity, DX, doc- accuracy, test-completeness) ran against the initial #677 commit; this commit lands everything that fits in the same PR plus follow-up issues for what doesn't. Correctness (security LOW + anti-pattern Critical): - middlewares/preset.go: the `json` template helper now escapes every RFC 8259 control byte (U+0000-U+001F), not just \, ", \n, \r, \t. The earlier hand-rolled escape would pass through NUL, BEL, FF, terminal ESC (0x1b, the start of every ANSI sequence), and the rest unchanged — strict JSON parsers on the receiving end then rejected the body. Real trigger: any job emitting progress bars, ls --color output, pytest, make. Implementation moved to a dedicated jsonStringEscape function with full round-trip regression test in TestWebhookTemplateFuncs_JSON_RFC8259EscapeCoverage. - middlewares/preset.go: the `truncate` helper is now rune-aware (utf8.RuneCountInString + range over s for the boundary). The byte-slice version would split multi-byte UTF-8 mid-rune when truncating non-ASCII output, producing a replacement rune that would combine with the json helper to break JSON parsing downstream. TestWebhookTemplateFuncs_Truncate now pins multi-byte emoji + ASCII mixes. - Added a new `jsonRaw` template helper backed by encoding/json.Marshal for future presets that want to emit a fully-formed JSON value rather than a string interior. Test e2e completeness (test-engineer review HIGH gaps): - cli/docker_labels_test.go: three new tests exercise the full Docker- label pipeline (buildFromDockerContainers → applyGlobalWebhookLabels → mergeWebhookGlobals → InitManager → NewWebhook) — previously only the programmatic path was end-to-end tested. - TestURLOnlyWebhookFromLabels_UsesJSONPostFallback: url-only label config resolves to json-post. - TestCustomDefaultPresetViaLabel_AppliesToURLOnlyWebhook: operator-chosen default via label flows through to NewWebhook. - TestDefaultPresetOptOutViaLabel_FailsURLOnlyWebhook: explicit empty label captured as non-nil "" (opt-out) and attach fails with the new actionable error. - middlewares/webhook_test.go: TestNewWebhook_URLOnly_UsesJSONPostFallback now asserts actual field VALUES in the JSON body (not just key presence) so a future template refactor that hard-codes a field fails loudly. Added TestJSONPostPreset_OptionalLinkBlock_RendersValidJSON exercising BOTH branches of the {{if .Preset.Link}} conditional — catches the comma-after-ofelia / no-trailing-comma landmine. DX (DX reviewer findings 1-6): - README.md: line 74 webhook description now mentions the url-only flow ("just `url = ...` to POST a JSON payload via the bundled `json-post` fallback"). The global-keys reference list around line 291 now documents `webhook-default-preset` alongside its siblings. - docs/webhooks.md: Quick Start now LEADS with "Simplest case: just POST to a URL" (json-post fallback example) before the preset-based examples. The Docker-label reference table now lists `ofelia.webhook-default-preset`. Opt-out section explains the WHY (audit policy, ban-implicit-fallback, BYO-default) before the WHAT. - middlewares/webhook.go: NewWebhook's opt-out error now names `webhook-default-preset` and the bundled fallback so operators can grep their way from a log line to the docs. The previous error ("either preset or url") didn't mention either knob. Continuity / upgrade impact (continuity reviewer Important): - CHANGELOG.md: [Unreleased] Added entry now LEADS with the upgrade-impact lede ("webhooks configured with `url = ...` but no `preset` were previously rejected at startup … they now attach and fire"). Operators reading release notes during upgrade see the behavior change first, the API details second. Follow-up issues filed for the four out-of-scope findings: - #678 — [global] unknown-key warning lacks "did you mean?" suggestion (parity with job-section path; pre-existing asymmetry). - #679 — Local preset file silently shadowed by bundled name; warn on AddLocalPresetDir collision (pre-existing precedence; PR #677 makes it more likely because `json-post` is a generic-enough name). - #673 (already open) — sendWithRetry's time.Sleep ignores ctx. - #674 (already open) — rebuildAllMiddlewares allocates fresh *http.Client per webhook per reconcile. Signed-off-by: Sebastian Mendel <[email protected]>
There was a problem hiding this comment.
Automated approval for maintainer PR
All automated quality gates passed. See SECURITY_CONTROLS.md for compensating controls.
Two follow-up Copilot review comments on PR #677: - middlewares/preset.go: DefaultPreset() godoc said the opt-out path fails attachment "when the per-webhook config also omits both `preset` and `url`". That was true before the actionable-error change in commit 57b4004 — now NewWebhook fails attachment whenever `preset` is empty after fallback, regardless of `url`, with the new error message that names `webhook-default-preset`. Comment now reflects the actual control-flow. - middlewares/uncovered_ext_test.go: the regression-test comment pointed at "the parallel reviewer pass on #677" — accurate in isolation but #677 is the PR, not the canonical issue. Re-pointed at #676 (the issue tracking the json-post feature) so future readers can navigate from the test to the design discussion. Signed-off-by: Sebastian Mendel <[email protected]>
There was a problem hiding this comment.
Automated approval for maintainer PR
All automated quality gates passed. See SECURITY_CONTROLS.md for compensating controls.
|
…u mean?" Pre-fix, the unknown-key warning was asymmetric across INI section types: job sections (job-exec, job-run, etc.) printed '(did you mean "..."?)' via findClosestMatch, but [global] and [docker] printed bare '(typo?)' with no suggestion — leaving the operator to read the line and guess. Surfaced in PR #677's parallel-reviewer pass: the new webhook-default-preset global key is easy to mistype (webhook-defauls-preset swaps t/s) and the typo logged a [global] warning with no hint at the right spelling. Fix extracts a logSectionUnknownKeyWarnings helper that mirrors logJobUnknownKeyWarnings's filename/suggestion matrix and call it from both [global] and [docker] paths in logUnknownKeyWarnings (file-based) and BuildFromString (string-based). Known keys come from extractMapstructureKeys(Config{}.Global) and extractMapstructureKeys(DockerConfig{}) via two thin helpers (globalKnownKeys / dockerKnownKeys) so the suggestion list cannot drift from the actual decoded struct. Three new tests: - TestGlobalSectionUnknownKeyWarning_DidYouMean: the issue's exact example — webhook-defauls-preset suggests webhook-default-preset. - TestGlobalSectionUnknownKeyWarning_NoSuggestion: locks the fallback to '(typo?)' when no close match exists. - TestDockerSectionUnknownKeyWarning_DidYouMean: same parity assertion for the [docker] path so a future code split can't silently drop suggestions on one half. Closes #678. Tracks against #621 (global-keys drift detection family). Signed-off-by: Sebastian Mendel <[email protected]>



Summary
Closes #676. Adds the bundled
json-postpreset and a[global] webhook-default-presetselector. After this PR, a webhook configured with justurl = ...actually works — Ofelia POSTs a JSON payload describing the execution to that URL, no custom preset YAML required.The docs section "Custom Webhooks" already promised this behavior, but the code rejected it:
loader.Load("")returned an error before any send code ran, and there was no bundled preset that turned a bare URL into a working webhook. The doc snippet has been false since the webhook system shipped.Three-state semantics (no sentinel ambiguity)
DefaultPresetis*stringso unset / explicit-empty / explicit-value are all distinguishable — mirrors theSaveConfig.RestoreHistorypattern inmiddlewares/save.go. Same conversation as #640 and #652 but resolved properly for this new field:"json-post"webhook-default-preset =(non-nil "")""(NewWebhook then fails Validate)webhook-default-preset = my-custom"my-custom"Verified end-to-end via gcfg INI parsing — see
TestEffectiveDefaultPreset_ThreeIntents.Resolution timing
The fallback is selected via
WebhookGlobalConfig.EffectiveDefaultPreset()at webhook attach time inNewWebhook, not at daemon startup. Live INI reloads and label changes towebhook-default-presettake effect on the next attach without restart.json-post payload shape
POST {url}withContent-Type: application/json; charset=utf-8, body:{ "job": {"name": "...", "command": "...", "schedule": "...", "type": "..."}, "execution": { "id": "...", "status": "...", "failed": false, "skipped": false, "duration": "1.234s", "duration_ns": 1234000000, "start_time": "...", "end_time": "...", "error": "", "output": "...", "stderr": "" }, "host": {"hostname": "...", "timestamp": "..."}, "ofelia": {"version": "..."} }outputandstderrare truncated to 4000 chars (test pins this). Optionallinkblock is included when the webhook config haslink =set. Every dynamic string field is wrapped through the templatejsonhelper so JSON-hostile chars (",\,\n,\r,\t) round-trip cleanly — exercised on the failure path byTestJSONPostPreset_FailedExecutionRendersValidJSON.Tests (every documented use case has an e2e or unit pin)
TestBundledPreset_JSONPostExistsjson-post,urlis the only required variable.TestEffectiveDefaultPreset_ThreeIntentsTestNewWebhook_URLOnly_UsesJSONPostFallbackTestNewWebhook_DefaultPresetCustomjson-post.TestNewWebhook_ExplicitPresetWinsOverDefaultpreset = Xoverrides any global default.TestNewWebhook_DefaultPresetOptOutDefaultPreset=&""restores the pre-fallback "either preset or url" error.TestNewWebhook_NilLoaderReturnsErrorTestJSONPostPreset_FailedExecutionRendersValidJSONTestJSONPostPreset_OutputTruncationTestApplyGlobalWebhookLabels_DefaultPresetTestMergeWebhookGlobals_DefaultPresetCoverage:
mergeWebhookGlobals,applyGlobalWebhookLabels,EffectiveDefaultPreset,PresetLoader.DefaultPresetare all at 100% statement coverage.Design notes worth flagging
json-postovergeneric:preset: json-postself-documents (method + body format), matches the existing hyphenated convention (ntfy-token), and avoidsgenerictautology.webhook-default-presetlives in the operator-tunable label allow-list. Not SSRF-sensitive: narrowing or widening the default cannot reach a new destination — the URL allow-list and preset loader still gate delivery.*stringfield inWebhookGlobalConfig. Sibling string globals (Webhooks,AllowedHosts) still use plain string + sentinel-compare inmergeWebhookGlobals— inconsistent on purpose.*stringis the right answer where unset / explicit-empty / explicit-value all matter; migrating siblings is a separate follow-up.Test plan
go test ./...(incl.cli,middlewares)golangci-lint run --timeout 3mgo test -coverprofile=... ./middlewares/... ./cli/...— all four new functions at 100%webhook-default-preset =(empty) restores the original validation error*stringverified for all three intents (omit / empty / explicit)