Skip to content

Conversation

@etspaceman
Copy link
Contributor

See https://github.com/etspaceman/kinesis-mock/releases/tag/0.3.5

In particular, fixes a couple of fields in response types so that they are nullable, rather than empty lists (or 0'd integers).

See https://github.com/etspaceman/kinesis-mock/releases/tag/0.3.5

In particular, fixes a couple of fields in response types so that they are nullable, rather than empty lists (or 0'd integers).
FailedRecordCount is now null, not 0, for these types of responses
@etspaceman etspaceman requested a review from calvernaz as a code owner February 7, 2023 16:23
"stream_description": {
"StreamDescription": {
"EncryptionType": "NONE",
"EnhancedMonitoring": [
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are not returned when empty now. According to the Kinesis API, ShardLevelMetrics must be non-empty if this is returned, otherwise it is null. In this case, it is null.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @etspaceman !

These are actually validated against AWS. Our *.snapshot.json files are all automatically generated when running against AWS and we generally don't update them manually for this reason. To update them we run the test with the following additional environment variables.

AWS_PROFILE=<your-profile>
TEST_TARGET=AWS_CLOUD
SNAPSHOT_UPDATE=1  # or set it to `0` if you just want to verify that the existing snapshot still validates against AWS

This could've changed since we recorded it, but I just updated to have the newest specs in boto3, re-ran the test and this seems to still be the observed result unfortunately.

Additionally, please note that these are not the raw HTTP responses you're seeing, but the objects generated by the SDK.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's very interesting. Here is section in the smithy documents about this:

https://github.com/aws/aws-sdk-js-v3/blob/main/codegen/sdk-codegen/aws-models/kinesis.json#L5072

You can see that the min length here is 1. I've confirmed this in the Java SDK as well.

But also, if I run a describe-stream in the aws-cli, I see empty lists being returned. So I wonder if this is a bug in Smithy.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dominikschubert I am going to open a support ticket with AWS on this one to get some clarity. For now I'll close this PR and revert my change in kinesis-mock

"describe_stream_result": {
"StreamDescription": {
"EncryptionType": "NONE",
"EnhancedMonitoring": [
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

result = requests.post(url, data=json.dumps(test_data))
result = json.loads(to_str(result.content))
assert 0 == result["FailedRecordCount"]
assert None == result.get["FailedRecordCount"]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is null now. According to the Kinesis API, FailedRecordCount must be > 0 if it is returned, otherwise it is null.

# by default, no errors
test_no_errors = kinesis.put_records(StreamName=stream_name, Records=records)
assert test_no_errors["FailedRecordCount"] == 0
assert test_no_errors.get["FailedRecordCount"] == None
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@etspaceman etspaceman closed this Feb 8, 2023
@etspaceman etspaceman deleted the patch-10 branch February 8, 2023 14:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants