[SFN] [TestState] Add mock result validation according to field validation modes #13419
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tiurin
added
the
semver: minor
tiurin
added
docs: skip
tiurin
changed the title
strict field validation mode
tiurin
changed the title
strict field validation modeSTRICT field validation mode
LocalStack Community integration with Pro 2 files ± 0 2 suites ±0 20m 32s ⏱️ - 1h 40m 17s Results for commit 3952452. ± Comparison against base commit 58f49e9. ♻️ This comment has been updated with latest results. |
Test Results (amd64) - Integration, Bootstrap 5 files ± 0 5 suites ±0 39m 6s ⏱️ - 2h 2m 8s Results for commit 3952452. ± Comparison against base commit 58f49e9. ♻️ This comment has been updated with latest results. |
tiurin
force-pushed
the
feature/step-functions/test-state-main
branch
from
7bf7ea6 to
c5d5aab
Compare
tiurin
force-pushed
the
feature/step-functions/test-state-field-validation-mode-strict
branch
from
957a2e0 to
08d99fe
Compare
gregfurman
approved these changes
Nov 27, 2025
Contributor
gregfurman
left a comment
gregfurman
left a comment
There was a problem hiding this comment.
LGTM. Nice one addressing these (frankly, extremely) complex validation cases!
Could be worth looking into using a pydantic model to easier validate this JSON like I did in:
| return test_program.test_state is not None | ||
|
|
||
| @staticmethod | ||
| def validate_mock(mock: TestStateMock, definition: Definition, state_name: StateName) -> None: |
Contributor
There was a problem hiding this comment.
ok wow. This is gnarly 💀
...core/localstack/services/stepfunctions/asl/static_analyser/test_state/test_state_analyser.py
Show resolved
Hide resolved
tiurin
changed the title
STRICT field validation mode
tiurin
force-pushed
the
feature/step-functions/test-state-field-validation-mode-strict
branch
from
48462aa to
3952452
Compare
TestState validates mocked responses against AWS API models with three validation modes: STRICT (this is the default and validates all required fields), PRESENT (validates field types and names), and NONE (no validation). STRICT field validation mode checks the presence of required fields and validates the type and format of the fields in the mock against the respective service shape in AWS API definitions. PRESENT field validation mode doesn't check the presence of required fields. It only validates the type and format of the fields that are present in the user-provided mock. NONE mode means no format validation will be performed. A check that the result is a valid JSON string is still performed even in NONE field validation mode. Note: fields that are not present in the API spec but are supplied in the mock will be allowed, because of forward-compatibility.
tiurin
force-pushed
the
feature/step-functions/test-state-field-validation-mode-strict
branch
from
3952452 to
06902e0
Compare
tiurin
merged commit 5eb1f18
into
feature/step-functions/test-state-main
12 checks passed
tiurin
deleted the
feature/step-functions/test-state-field-validation-mode-strict
branch
tiurin
added a commit
that referenced
this pull request
Nov 28, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
TestState validates mocked responses against AWS API models with three validation modes: STRICT (this is the default and validates all required fields), PRESENT (validates field types and names), and NONE (no validation).
STRICTfield validation mode checks the presence of required fields and validates the type and format of the fields in the mock against the respective service shape in AWS API definitions.PRESENTfield validation mode doesn't check the presence of required fields. It only validates the type and format of the fields that are present in the user-provided mock.NONEmode means no format validation will be performed. A check that the result is a valid JSON string is still performed even in NONE field validation mode.Note: fields that are not present in the API spec but are supplied in the mock will be allowed, because of forward-compatibility.
Closes DRG-220.
Changes
Add mock result validation method to a static analyzer that already contains several other validations.
Limitations:
range/length validation is not implemented yet (added DRG-258 as a follow-up)
Tests
Related