Skip to content

Commit e56c2c7

Browse files
openapi3: aggregate independent validation errors via EnableMultiError (#1185)
* openapi3: aggregate independent validation errors via EnableMultiError Today Validate returns the first error it encounters and stops, so a spec with several independent defects only reports one at a time: fix, re-run, see the next, repeat. With the new EnableMultiError ValidationOption, container validators aggregate independent problems and return them together as a flat MultiError. Aggregation happens at the container fan-out points where it is safe to continue past an error: T, Paths, PathItem, Operation, Components, Responses. Leaf validators (Schema, Info, Server, etc.) still fail fast, since continuing past a leaf error can hit a nil deref. The result shape is flat: each top-level MultiError entry is one fully-wrapped chain (SectionValidationError -> PathValidationError -> OperationValidationError -> typed leaf). Consumers iterating the MultiError get one entry per problem; consumers using errors.As / errors.Is work unchanged thanks to MultiError.As / MultiError.Is. By default Validate returns the first error encountered, byte-identical to today: the existing test suite passes through the same code path it always did. * openapi3: keep "return validateExtensions" pattern in container validators CI greps for "return .*validateExtensions" in openapi3/ and asserts the count equals the number of Extensions fields. The previous commit moved those calls to "me.emit(validateExtensions(...))" + "return me.result()", breaking the count. Add a finalize helper on errCollector that emits its argument and returns the accumulated result, so each container's trailing line is a single "return me.finalize(validateExtensions(ctx, X.Extensions))" that both reads cleanly and matches the CI fence regex. * openapi3: address review feedback on EnableMultiError Top-level: - cmd/validate: add --multi flag (mirrors --defaults / --examples / --patterns) - Convert simple leaves to aggregate independent errors: License, Info, Server (singular + plural), Tag (singular + plural), ExternalDocs. Each is sequential independent checks with no guard-then-deref pattern, structurally identical to the existing container validators. Schema, Parameter, MediaType have internal cross-field dependencies and are left for follow-up PRs after per-method analysis. - Rewrite EnableMultiError docstring: drop the explicit fan-out list (would go stale), name the transition explicitly, explain why some leaves are not yet converted, and replace the generic errors.As / errors.Is note with a concrete how-to. Inline: - validate_multi_error_test.go: use require.ErrorContains for chain checks; tighten GreaterOrEqual to Equal where the count is exact. - response.go: route Responses.Validate's empty-responses error through me.emit for consistency with the rest of the multi-error path. - components.go, paths.go, openapi3.go: add comments at the error-path continue statements explaining why we skip descending past a malformed parent key (component name, path key, nil webhook). - error_collector.go (new): move the errCollector type and methods out of validation_options.go so the options file stays focused on options. - openapi3.go: revert to the wrap-variable reassignment pattern per section block; keep wrapSection as the factory. * test: hoist err.Error() out of require.Equal to satisfy CI grep fence * openapi3: wrap plural examples Validate errors in newSchemaValueError Parameter.Validate and MediaType.Validate already use newSchemaValueError for the singular `example:` field, but the plural `examples:` map loop returned a bare fmt.Errorf wrapping the underlying *SchemaError. That meant errors.As(err, &SchemaValueError) didn't match for plural examples, so downstream consumers fell back to a generic catch-all instead of recognising the example-violates-schema cluster. Wrap the plural-loop result in newSchemaValueError("example", ...) too, preserving the example key inside the wrap so the rendered message still names the offending example. Aligns the plural path with the singular path; both now surface as *SchemaValueError to errors.As consumers. Existing example_validation_test.go assertions and the apis_guru golden fixtures gain the "invalid example: " prefix the wrap adds. * openapi3: point Examples Validate failures at the example's value, not the parent For plural Examples entries on Parameter/MediaType, the Validate failure used the parent struct's Origin, which deep-links to the parameter or media-type start (e.g. the `in: query` line) rather than the offending example value. Build the SchemaValueError's Origin from the Example struct's per-field origin (`Origin.Fields["value"]`) when available, falling back to the example's struct origin, then to the parent. No message-format change: existing string assertions continue to hold, only the resolved line/column tightens to the example's `value:` field. * openapi3: keep validating extensions when responses is empty The empty-responses path used 'return me.result()' which short-circuited before validateExtensions could run, so extension errors would never aggregate with the empty-responses finding under multi mode. Drop the early return and let control fall through to the trailing finalize() call; the for loop is a no-op on an empty Responses, so the only behaviour change is that validateExtensions now runs. Spotted by @fenollp in PR review. * test: regression pin for the empty-responses + extension aggregation fix Adds TestResponses_Validate_EmptyAndExtensionAggregate which constructs an empty Responses with a non-x- sibling on Extensions and asserts both findings surface under EnableMultiError. Reverting the previous commit's fall-through edit causes this test to fail (one leaf instead of two).
1 parent 7ea1ac8 commit e56c2c7

76 files changed

Lines changed: 644 additions & 226 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/docs/openapi3.txt‎

Lines changed: 16 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -419,7 +419,7 @@ func (components Components) MarshalYAML() (any, error)
419419
func (components *Components) UnmarshalJSON(data []byte) error
420420
UnmarshalJSON sets Components to a copy of data.
421421

422-
func (components *Components) Validate(ctx context.Context, opts ...ValidationOption) (err error)
422+
func (components *Components) Validate(ctx context.Context, opts ...ValidationOption) error
423423
Validate returns an error if Components does not comply with the OpenAPI
424424
spec.
425425

@@ -2714,7 +2714,7 @@ func (server Server) ParameterNames() ([]string, error)
27142714
func (server *Server) UnmarshalJSON(data []byte) error
27152715
UnmarshalJSON sets Server to a copy of data.
27162716

2717-
func (server *Server) Validate(ctx context.Context, opts ...ValidationOption) (err error)
2717+
func (server *Server) Validate(ctx context.Context, opts ...ValidationOption) error
27182718
Validate returns an error if Server does not comply with the OpenAPI spec.
27192719

27202720
type ServerURLMismatchedBraces struct{ ValidationError }
@@ -3162,6 +3162,20 @@ func EnableExamplesValidation() ValidationOption
31623162
EnableExamplesValidation does the opposite of DisableExamplesValidation.
31633163
By default, all schema examples are validated.
31643164

3165+
func EnableMultiError() ValidationOption
3166+
EnableMultiError makes Validate aggregate independent validation errors and
3167+
return them together as a MultiError instead of returning the first error
3168+
and stopping. By default, Validate is fail-fast.
3169+
3170+
Not every validator reports more than one error yet. Some, such as Schema,
3171+
run checks that build on earlier ones, so continuing past a failure can
3172+
hit a nil dereference or produce nonsense secondary errors. We will keep
3173+
converting more validators in follow-up changes as each one is analyzed.
3174+
3175+
To pull a specific error type out of the result, use errors.As(err,
3176+
&target). It walks into the MultiError automatically, so the same call works
3177+
whether Validate returned one error or many.
3178+
31653179
func EnableSchemaDefaultsValidation() ValidationOption
31663180
EnableSchemaDefaultsValidation does the opposite of
31673181
DisableSchemaDefaultsValidation. By default, schema default values are

‎cmd/validate/main.go‎

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,11 +32,16 @@ var (
3232
patterns = flag.Bool("patterns", defaultPatterns, "when false, allows schema patterns unsupported by the Go regexp engine")
3333
)
3434

35+
var (
36+
defaultMulti = false
37+
multi = flag.Bool("multi", defaultMulti, "when true, aggregate independent validation errors instead of returning the first one")
38+
)
39+
3540
func main() {
3641
flag.Parse()
3742
filename := flag.Arg(0)
3843
if len(flag.Args()) != 1 || filename == "" {
39-
log.Fatalf("Usage: go run github.com/getkin/kin-openapi/cmd/validate@latest [--defaults] [--examples] [--ext] [--patterns] -- <local YAML or JSON file>\nGot: %+v\n", os.Args)
44+
log.Fatalf("Usage: go run github.com/getkin/kin-openapi/cmd/validate@latest [--defaults] [--examples] [--ext] [--patterns] [--multi] -- <local YAML or JSON file>\nGot: %+v\n", os.Args)
4045
}
4146

4247
data, err := os.ReadFile(filename)
@@ -77,6 +82,9 @@ func main() {
7782
if !*patterns {
7883
opts = append(opts, openapi3.DisableSchemaPatternValidation())
7984
}
85+
if *multi {
86+
opts = append(opts, openapi3.EnableMultiError())
87+
}
8088

8189
if err = doc.Validate(loader.Context, opts...); err != nil {
8290
log.Fatalln("Validation error:", err)
@@ -96,6 +104,9 @@ func main() {
96104
if *patterns != defaultPatterns {
97105
log.Fatal("Flag --patterns is only for OpenAPIv3")
98106
}
107+
if *multi != defaultMulti {
108+
log.Fatal("Flag --multi is only for OpenAPIv3")
109+
}
99110

100111
var doc openapi2.T
101112
if _, err := yaml.Unmarshal(data, &doc, yaml.DecodeOpts{DisableTimestamps: true}); err != nil {

‎openapi3/components.go‎

Lines changed: 60 additions & 74 deletions
Original file line numberDiff line numberDiff line change
@@ -108,100 +108,86 @@ func (components *Components) UnmarshalJSON(data []byte) error {
108108
}
109109

110110
// Validate returns an error if Components does not comply with the OpenAPI spec.
111-
func (components *Components) Validate(ctx context.Context, opts ...ValidationOption) (err error) {
111+
func (components *Components) Validate(ctx context.Context, opts ...ValidationOption) error {
112112
ctx = WithValidationOptions(ctx, opts...)
113-
114-
for _, k := range componentNames(components.Schemas) {
115-
v := components.Schemas[k]
116-
if err = ValidateIdentifier(k); err != nil {
117-
return fmt.Errorf("schema %q: %w", k, err)
118-
}
119-
if err = v.Validate(ctx); err != nil {
120-
return fmt.Errorf("schema %q: %w", k, err)
113+
me := newErrCollector(ctx)
114+
115+
validateMap := func(label string, names []string, validate func(k string) error) error {
116+
for _, k := range names {
117+
if idErr := ValidateIdentifier(k); idErr != nil {
118+
if err := me.emit(fmt.Errorf("%s %q: %w", label, k, idErr)); err != nil {
119+
return err
120+
}
121+
// Skip validating the component's value when its name is
122+
// invalid: any leaf error from validate(k) would surface as
123+
// "<bad-name>: <leaf-error>" and has no resolution path
124+
// until the name is fixed. The continue keeps the noise
125+
// per component bounded to a single, actionable finding.
126+
continue
127+
}
128+
wrap := func(e error) error { return fmt.Errorf("%s %q: %w", label, k, e) }
129+
if err := me.emitWrapped(wrap, validate(k)); err != nil {
130+
return err
131+
}
121132
}
133+
return nil
122134
}
123135

124-
for _, k := range componentNames(components.Parameters) {
125-
v := components.Parameters[k]
126-
if err = ValidateIdentifier(k); err != nil {
127-
return fmt.Errorf("parameter %q: %w", k, err)
128-
}
129-
if err = v.Validate(ctx); err != nil {
130-
return fmt.Errorf("parameter %q: %w", k, err)
131-
}
136+
if err := validateMap("schema", componentNames(components.Schemas), func(k string) error {
137+
return components.Schemas[k].Validate(ctx)
138+
}); err != nil {
139+
return err
132140
}
133141

134-
for _, k := range componentNames(components.RequestBodies) {
135-
v := components.RequestBodies[k]
136-
if err = ValidateIdentifier(k); err != nil {
137-
return fmt.Errorf("request body %q: %w", k, err)
138-
}
139-
if err = v.Validate(ctx); err != nil {
140-
return fmt.Errorf("request body %q: %w", k, err)
141-
}
142+
if err := validateMap("parameter", componentNames(components.Parameters), func(k string) error {
143+
return components.Parameters[k].Validate(ctx)
144+
}); err != nil {
145+
return err
142146
}
143147

144-
for _, k := range componentNames(components.Responses) {
145-
if err = ValidateIdentifier(k); err != nil {
146-
return fmt.Errorf("response %q: %w", k, err)
147-
}
148-
v := components.Responses[k]
149-
if err = v.Validate(ctx); err != nil {
150-
return fmt.Errorf("response %q: %w", k, err)
151-
}
148+
if err := validateMap("request body", componentNames(components.RequestBodies), func(k string) error {
149+
return components.RequestBodies[k].Validate(ctx)
150+
}); err != nil {
151+
return err
152152
}
153153

154-
for _, k := range componentNames(components.Headers) {
155-
v := components.Headers[k]
156-
if err = ValidateIdentifier(k); err != nil {
157-
return fmt.Errorf("header %q: %w", k, err)
158-
}
159-
if err = v.Validate(ctx); err != nil {
160-
return fmt.Errorf("header %q: %w", k, err)
161-
}
154+
if err := validateMap("response", componentNames(components.Responses), func(k string) error {
155+
return components.Responses[k].Validate(ctx)
156+
}); err != nil {
157+
return err
162158
}
163159

164-
for _, k := range componentNames(components.SecuritySchemes) {
165-
v := components.SecuritySchemes[k]
166-
if err = ValidateIdentifier(k); err != nil {
167-
return fmt.Errorf("security scheme %q: %w", k, err)
168-
}
169-
if err = v.Validate(ctx); err != nil {
170-
return fmt.Errorf("security scheme %q: %w", k, err)
171-
}
160+
if err := validateMap("header", componentNames(components.Headers), func(k string) error {
161+
return components.Headers[k].Validate(ctx)
162+
}); err != nil {
163+
return err
172164
}
173165

174-
for _, k := range componentNames(components.Examples) {
175-
v := components.Examples[k]
176-
if err = ValidateIdentifier(k); err != nil {
177-
return fmt.Errorf("example %q: %w", k, err)
178-
}
179-
if err = v.Validate(ctx); err != nil {
180-
return fmt.Errorf("example %q: %w", k, err)
181-
}
166+
if err := validateMap("security scheme", componentNames(components.SecuritySchemes), func(k string) error {
167+
return components.SecuritySchemes[k].Validate(ctx)
168+
}); err != nil {
169+
return err
182170
}
183171

184-
for _, k := range componentNames(components.Links) {
185-
v := components.Links[k]
186-
if err = ValidateIdentifier(k); err != nil {
187-
return fmt.Errorf("link %q: %w", k, err)
188-
}
189-
if err = v.Validate(ctx); err != nil {
190-
return fmt.Errorf("link %q: %w", k, err)
191-
}
172+
if err := validateMap("example", componentNames(components.Examples), func(k string) error {
173+
return components.Examples[k].Validate(ctx)
174+
}); err != nil {
175+
return err
192176
}
193177

194-
for _, k := range componentNames(components.Callbacks) {
195-
v := components.Callbacks[k]
196-
if err = ValidateIdentifier(k); err != nil {
197-
return fmt.Errorf("callback %q: %w", k, err)
198-
}
199-
if err = v.Validate(ctx); err != nil {
200-
return fmt.Errorf("callback %q: %w", k, err)
201-
}
178+
if err := validateMap("link", componentNames(components.Links), func(k string) error {
179+
return components.Links[k].Validate(ctx)
180+
}); err != nil {
181+
return err
182+
}
183+
184+
if err := validateMap("callback", componentNames(components.Callbacks), func(k string) error {
185+
return components.Callbacks[k].Validate(ctx)
186+
}); err != nil {
187+
return err
202188
}
203189

204-
return validateExtensions(ctx, components.Extensions)
190+
return me.finalize(validateExtensions(ctx, components.Extensions))
205191
}
206192

207193
var _ jsonpointer.JSONPointable = (*Schemas)(nil)

‎openapi3/error_collector.go‎

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
package openapi3
2+
3+
import "context"
4+
5+
// errCollector aggregates validation errors inside a Validate method.
6+
//
7+
// When multi-error mode is enabled (EnableMultiError), emit records the error
8+
// and returns nil so the caller continues to the next sibling; if the error is
9+
// itself a MultiError, its leaves are appended individually so the result is a
10+
// flat MultiError of fully-wrapped problems (this matches what most consumers
11+
// expect: one MultiError entry per independent problem).
12+
//
13+
// When multi-error mode is off, emit returns the error unchanged so the caller
14+
// fails fast, preserving the historical behavior byte-for-byte.
15+
//
16+
// emitWrapped applies wrap to err, distributing wrap over each leaf when err
17+
// is a MultiError. This is how validators attach per-section / per-path /
18+
// per-operation context to each aggregated leaf.
19+
//
20+
// result returns the accumulated MultiError, or nil if none were recorded.
21+
type errCollector struct {
22+
multi bool
23+
errs MultiError
24+
}
25+
26+
func newErrCollector(ctx context.Context) *errCollector {
27+
return &errCollector{multi: getValidationOptions(ctx).multiErrorEnabled}
28+
}
29+
30+
func (c *errCollector) emit(err error) error {
31+
if err == nil {
32+
return nil
33+
}
34+
if !c.multi {
35+
return err
36+
}
37+
if me, ok := err.(MultiError); ok {
38+
for _, sub := range me {
39+
if e := c.emit(sub); e != nil {
40+
return e
41+
}
42+
}
43+
return nil
44+
}
45+
c.errs = append(c.errs, err)
46+
return nil
47+
}
48+
49+
func (c *errCollector) emitWrapped(wrap func(error) error, err error) error {
50+
if err == nil {
51+
return nil
52+
}
53+
if !c.multi {
54+
return wrap(err)
55+
}
56+
if me, ok := err.(MultiError); ok {
57+
for _, sub := range me {
58+
if e := c.emitWrapped(wrap, sub); e != nil {
59+
return e
60+
}
61+
}
62+
return nil
63+
}
64+
return c.emit(wrap(err))
65+
}
66+
67+
func (c *errCollector) result() error {
68+
if len(c.errs) > 0 {
69+
return c.errs
70+
}
71+
return nil
72+
}
73+
74+
// finalize emits err (typically the last sibling validation in a container,
75+
// e.g. the extensions check) and returns the accumulated result. It collapses
76+
// the trailing emit-then-result pattern into a single line at each call site.
77+
func (c *errCollector) finalize(err error) error {
78+
if e := c.emit(err); e != nil {
79+
return e
80+
}
81+
return c.result()
82+
}

‎openapi3/example_validation_test.go‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ func TestExamplesSchemaValidation(t *testing.T) {
2929
param1example:
3030
value: abcd
3131
`,
32-
errContains: `invalid paths: invalid path /user: invalid operation POST: param1example`,
32+
errContains: `invalid paths: invalid path /user: invalid operation POST: invalid example: param1example`,
3333
},
3434
{
3535
name: "valid_parameter_examples",
@@ -67,7 +67,7 @@ func TestExamplesSchemaValidation(t *testing.T) {
6767
email: bad
6868
password: short
6969
`,
70-
errContains: `invalid paths: invalid path /user: invalid operation POST: example BadUser`,
70+
errContains: `invalid paths: invalid path /user: invalid operation POST: invalid example: example BadUser`,
7171
},
7272
{
7373
name: "valid_component_examples",
@@ -178,7 +178,7 @@ func TestExamplesSchemaValidation(t *testing.T) {
178178
password: password
179179
user_id: 4321
180180
`,
181-
errContains: `ReadWriteOnlyRequest: readOnly property "user_id" in request`,
181+
errContains: `invalid example: example ReadWriteOnlyRequest: readOnly property "user_id" in request`,
182182
},
183183
{
184184
name: "invalid_writeonly_response_examples",
@@ -195,7 +195,7 @@ func TestExamplesSchemaValidation(t *testing.T) {
195195
user_id: 4321
196196
`,
197197

198-
errContains: `ReadWriteOnlyResponse: writeOnly property "password" in response`,
198+
errContains: `invalid example: example ReadWriteOnlyResponse: writeOnly property "password" in response`,
199199
},
200200
}
201201

‎openapi3/external_docs.go‎

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -60,13 +60,18 @@ func (e *ExternalDocs) UnmarshalJSON(data []byte) error {
6060
// Validate returns an error if ExternalDocs does not comply with the OpenAPI spec.
6161
func (e *ExternalDocs) Validate(ctx context.Context, opts ...ValidationOption) error {
6262
ctx = WithValidationOptions(ctx, opts...)
63+
me := newErrCollector(ctx)
6364

6465
if e.URL == "" {
65-
return newExternalDocsURLRequired(e.Origin)
66+
if err := me.emit(newExternalDocsURLRequired(e.Origin)); err != nil {
67+
return err
68+
}
6669
}
6770
if _, err := url.Parse(e.URL); err != nil {
68-
return fmt.Errorf("url is incorrect: %w", err)
71+
if err := me.emit(fmt.Errorf("url is incorrect: %w", err)); err != nil {
72+
return err
73+
}
6974
}
7075

71-
return validateExtensions(ctx, e.Extensions)
76+
return me.finalize(validateExtensions(ctx, e.Extensions))
7277
}

0 commit comments

Comments
 (0)