Skip to content

Conversation

@kiwamizamurai
Copy link
Contributor

Problem

Currently, when a pattern validation is added to a previously unrestricted parameter or property, oasdiff incorrectly classifies this as a WARN level change instead of an ERROR level breaking change.

This is problematic because:

  • Adding a pattern restriction to a parameter that previously accepted any string value is a breaking change
  • Existing clients that were sending values not matching the new pattern will start failing
  • The current warning message is misleading and doesn't clearly indicate the severity

Example

Consider this OpenAPI change:

Before (example_before.yaml):

parameters:
  - name: cursor
    in: query
    description: Pagination cursor
    schema:
      type: string  # Accepts any string
      nullable: true
    required: false

After (example_after.yaml):

parameters:
  - name: cursor
    in: query
    description: Pagination cursor
    schema:
      type: string
      pattern: "^\\d+$"  # Now only accepts digits
      nullable: true
    required: false

This change will break existing clients that were sending non-digit cursor values, but was previously only flagged as a warning.

Solution

This PR implements proper severity classification for pattern-related changes:

  1. Pattern Addition (pattern added): ERROR level

    • When a pattern is added to a previously unrestricted parameter/property
    • New specific error message: "This is a breaking change because adding a pattern restriction to a previously unrestricted parameter will reject values that were previously accepted, breaking existing clients"
  2. Pattern Change (pattern changed): WARN level (unchanged)

    • When an existing pattern is changed to a different pattern
    • Keeps existing warning message since it's difficult to automatically determine if the new pattern is a superset
  3. Pattern Generalization (pattern generalized): INFO level (unchanged)

    • When a pattern is changed to a more permissive one (e.g., to .*)
  4. Pattern Removal (pattern removed): INFO level (unchanged)

    • When a pattern restriction is removed entirely

Changes Made

  • Updated severity rules in checker/rules.go:
    • RequestParameterPatternAddedId: WARNERROR
    • RequestPropertyPatternAddedId: WARNERROR
  • Added new localized error message pattern-added-error-comment for pattern additions
  • Updated existing warning message pattern-changed-warn-comment to be more comprehensive
  • Added multi-language support (English, Portuguese, Russian)
  • Updated all relevant tests to reflect the new severity levels
  • Fixed test ordering issues caused by the severity level changes

Verification

# Test with the provided example
make build
./bin/oasdiff breaking example_before.yaml example_after.yaml

# Expected output:
# 1 changes: 1 error, 0 warning, 0 info
# error [request-parameter-pattern-added] at example_after.yaml
#   in API GET /users
#     added the pattern '^\d+$' to the 'query' request parameter 'cursor'
#     This is a breaking change because adding a pattern restriction to a previously unrestricted parameter will reject values that were previously accepted, breaking existing clients

Backward Compatibility

This change only affects the severity classification of pattern additions. The detection logic remains the same, ensuring that:

  1. No previously detected changes are missed
  2. Only the severity level and error message change for pattern additions
  3. All other pattern-related changes maintain their existing behavior

- Add new error message for pattern additions in English
- Add Portuguese (Brazil) translation
- Add Russian translation
- Update pattern-changed-warn-comment to be more comprehensive
Regenerate localizations.go after adding pattern-added-error-comment
- Add PatternAddedCommentId constant for pattern addition errors
- Use PatternAddedCommentId for pattern additions instead of generic PatternChangedCommentId
- Keep PatternChangedCommentId for pattern changes (WARN level)
BREAKING CHANGE: Pattern additions are now classified as ERROR level instead of WARN level

- RequestParameterPatternAddedId: WARN -> ERR
- RequestPropertyPatternAddedId: WARN -> ERR

This correctly identifies pattern additions as breaking changes since they
restrict previously accepted values.
- Update pattern addition tests to expect ERR level instead of WARN
- Update test assertions for new error messages
- Verify pattern changes still use WARN level
Pattern additions now have ERR level priority, changing the order of
breaking changes in test results. Update expected order in:
- TestBreaking_OperationID
- TestBreaking_LinkOperationID
@kiwamizamurai
Copy link
Contributor Author

@reuvenharrison please review this pr

config,
[]any{patternDiff.To, paramLocation, paramName},
PatternChangedCommentId,
PatternAddedCommentId,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice catch

@codecov-commenter
Copy link

codecov-commenter commented Jul 3, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 87.58%. Comparing base (ab901ac) to head (64e7497).
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #720   +/-   ##
=======================================
  Coverage   87.58%   87.58%           
=======================================
  Files         241      241           
  Lines       13956    13956           
=======================================
  Hits        12224    12224           
  Misses       1311     1311           
  Partials      421      421           
Flag Coverage Δ
unittests 87.58% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@reuvenharrison reuvenharrison merged commit d68743c into oasdiff:main Jul 3, 2025
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants