Aws http event schema#8301
Conversation
Codecov Report
@@ Coverage Diff @@
## master #8301 +/- ##
==========================================
- Coverage 88.01% 87.95% -0.07%
==========================================
Files 249 249
Lines 9307 9255 -52
==========================================
- Hits 8192 8140 -52
Misses 1115 1115
Continue to review full report at Codecov.
|
I think it's a good call, unless there are some properties which could be configured in both contexts, then I would also add them to |
|
Questions:
|
|
@thewizarodofoz thanks for questions. I know it's a challenging one.
I think we may skip on that, and just accept whatever string. Trying to be strict on such detached things (as I see this is setn as into some string formartted config to AWS), may result with unwanted maintenance cost,
I see what you mean. That if it's an object with no |
After we have #8319 in, there's no need for I wouldn't generalize |
Can you elaborate a bit more? Anyway I would stay away from if/else, as it'll make reported errors obscure and not well informative. I think it's ok to implement more complex validation rules inline and let errors with meaningful messages surface |
|
@medikoo ready for review |
medikoo
left a comment
There was a problem hiding this comment.
Thank you @thewizarodofoz ! Please see my comments, I think schema for object version has one important flaw (I decided to pause my review until we clarify that)
| // TODO: Complete schema, see https://github.com/serverless/serverless/issues/8018 | ||
| anyOf: [{ type: 'string' }, { type: 'object' }], | ||
| oneOf: [ | ||
| { type: 'string', pattern: `^(${httpMethods.join('|')}) \\S+` }, |
There was a problem hiding this comment.
I think we need to use regex to be case insensitive (same as we did in HTTP API:
Note that supporting just all lower case and all upper case will probably be not good enoug, as I take Get or Post also works
There was a problem hiding this comment.
Do you think we should have the same case-insensitive treatment (using regex) for similar cases like authorizer.type and integration?
There was a problem hiding this comment.
I think whenever we need to use regexp pattern and expect case insensitive matching, we should use regex instead of pattern
So if it's the case for authorizer.type an integration then, yes
| method: { | ||
| enum: extendWithUppercase(httpMethods), | ||
| }, | ||
| path: { type: 'string' }, |
There was a problem hiding this comment.
It'll be nice to use same regex as in httpApi event
| { type: 'string', pattern: `^(${httpMethods.join('|')}) \\S+` }, | ||
| baseIntegrationSchema, | ||
| awsIntegrationSchema, | ||
| cognitoIntegrationSchema, | ||
| httpIntegrationSchema, | ||
| httpVpcLinkIntegrationSchema, | ||
| lambdaIntegrationSchema, | ||
| lambdaProxyIntegrationSchema, | ||
| mockIntegrationSchema, |
There was a problem hiding this comment.
I believe we expect method and path unconditionally, while above configuration, seem to expect different set of properties exclusively (and most of them do not require method and path), that doesn't look right.
In my understanding, schema for http is either string (in form of method path), or single version of object where method and path are required.
In similar manner as in httpApi event here:
serverless/lib/plugins/aws/package/compile/events/httpApi/index.js
Lines 57 to 79 in 5851bca
but ofc we will have more properties in case of this event
There was a problem hiding this comment.
I declared path as required in baseIntegrationSchema, so all other integration schemas inherit that.
I did fail to declare method as required as well.
The reason I used multiple object schemas instead of one is there many conditions regarding integration types. That is what I referred to when I wrote about if/else - if I go with one object for all integration types, then I would have to use many if/else clauses to declare all constraints.
There was a problem hiding this comment.
Indeed, sorry I failed to see, that other objects, share all properties from baseIntegrationSchema
medikoo
left a comment
There was a problem hiding this comment.
Thank you @thewizarodofoz! It's a very challenging one, and what you've propsed looks very promising.
Please see my comments
| oneOf: [ | ||
| { type: 'string' }, | ||
| caseInsensitive('aws_iam'), | ||
| { $ref: '#/definitions/awsArn' }, |
There was a problem hiding this comment.
I think it should be awsArnString
| oneOf: ['token', 'cognito_user_pools', 'request', 'aws_iam'].map(caseInsensitive), | ||
| }, | ||
| authorizerId: { | ||
| oneOf: [{ type: 'string' }, { $ref: '#/definitions/awsCfRef' }], |
There was a problem hiding this comment.
Why not simply awsCfInstruction? Are we sure that relying on any other, aside of Ref, intrinsic functions is not supported here?
| oneOf: [{ type: 'string' }, { $ref: '#/definitions/awsCfRef' }], | ||
| }, | ||
| arn: { $ref: '#/definitions/awsArn' }, | ||
| resultTtlInSeconds: { type: 'integer', maximum: 3600 }, |
There was a problem hiding this comment.
I think minimum should be 0 because:
If you specify a value greater than 0, API Gateway caches the authorizer responses
Meaning, setting 0 will disable cache. I have no experience with this, but I guess it's a valid option.
There was a problem hiding this comment.
Sure, my point was, to not welcome negative values, we should set minimum: 0 I believe
| identityValidationExpression: { type: 'string' }, | ||
| managedExternally: { type: 'boolean' }, | ||
| name: { type: 'string' }, | ||
| }, |
There was a problem hiding this comment.
Let's keep alphabetical order
| properties: { | ||
| allowCredentials: { type: 'boolean' }, | ||
| headers: { type: 'array', items: { type: 'string' } }, | ||
| maxAge: { type: 'integer', minimum: 0 }, |
There was a problem hiding this comment.
I think it should be minimum 1
| baseIntegrationSchema, | ||
| awsIntegrationSchema, | ||
| cognitoIntegrationSchema, | ||
| httpIntegrationSchema, | ||
| httpVpcLinkIntegrationSchema, | ||
| lambdaIntegrationSchema, | ||
| lambdaProxyIntegrationSchema, | ||
| mockIntegrationSchema, |
There was a problem hiding this comment.
One problem with this kind of configuration is that eventual errors will not be informative.
afaik if I declare method: FOO in otherwise perfectly valid configuration, the result error will be: "Unsupported configuration format", with no hint that error is at method property.
It goes from fact, that with such schema it's challenging to resolve which exactly oneOf option was chosen by user. AJV will expose errors resolved for each option, showing all of them will produce unbearable noise (we intentionally convert it then, to one "Unsupported configuration format")
We already had similar cases before, and we decided to configure properties for all options in one level, and leave (or introduce) inline validation that's meaningful for user as e.g. follows:
if (integrationType !== X && authorizer.claims) {
throw new ServelressError("authorizer.claims is not supported for given integration type");
}I would also stick here to that.
I believe we should not try to validate any complex interchangeable rules with JSON schema, as it then it becomes a maintenance burden we can't show a meaningful error message to user.
There was a problem hiding this comment.
I understand.
I will refactor this PR to have a single schema, and leave the validations needed for the edge cases.
medikoo
left a comment
There was a problem hiding this comment.
Thank you @thewizarodofoz, that looks very good! I've found few last potentials issues. Otherwise it seems ready to go
| async: { type: 'boolean' }, | ||
| authorizer: authorizerSchema, | ||
| connectionId: { type: 'string' }, | ||
| connectionType: { enum: ['vpc-link', 'VPC_LINK'] }, |
There was a problem hiding this comment.
We need to support both forms in case insensitive way
| passThrough: { enum: ['NEVER', 'WHEN_NO_MATCH', 'WHEN_NO_TEMPLATES'] }, | ||
| schema: { | ||
| type: 'object', | ||
| additionalProperties: { type: ['object', 'string'] }, |
There was a problem hiding this comment.
On AWS side it's type Json, does it mean that both forms are accepted, object and string ?
There was a problem hiding this comment.
I got confused by the example:
functions:
create:
handler: posts.create
events:
- http:
path: posts/create
method: post
request:
schema:
application/json: ${file(create_request.json)}I understand now that ${file(create_request.json)} resolves to json (as object) so the string option is not relevant
| oneOf: [ | ||
| { properties: { origin: { type: 'string' } } }, | ||
| { | ||
| properties: { | ||
| origins: { | ||
| type: 'array', | ||
| items: { type: 'string' }, | ||
| }, | ||
| }, | ||
| }, | ||
| ], |
There was a problem hiding this comment.
As I test locally, in this form schema will in most cases fail, we've used oneOf, and no property listed is required (so in most cases both variants match, and that's invalid by schema).
I guess it's not exposed by tests, only because there are no tests that involve this event written with runServerless
Normally we configure such things, by defining involved properties normally in properties section , and use oneOf to configure two different required collections
Codecov Report
@@ Coverage Diff @@
## master #8301 +/- ##
==========================================
+ Coverage 88.01% 88.03% +0.01%
==========================================
Files 249 249
Lines 9307 9267 -40
==========================================
- Hits 8192 8158 -34
+ Misses 1115 1109 -6
Continue to review full report at Codecov.
|
medikoo
left a comment
There was a problem hiding this comment.
Thank you @thewizarodofoz! We're closer! :)
| async: { type: 'boolean' }, | ||
| authorizer: authorizerSchema, | ||
| connectionId: { type: 'string' }, | ||
| connectionType: { enum: ['vpc-link', 'VPC_LINK'].map(caseInsensitive) }, |
There was a problem hiding this comment.
I think this produces as schema, where we expect an objects with regexp properties.
I believe this should be converted to anyOf list
There was a problem hiding this comment.
I used oneOf to be consistent with other places that have .map(caseInsensitive). The array will have distinct patterns, so a value can never match more than one option.
| type: 'object', | ||
| propertyNames: { | ||
| type: 'string', | ||
| pattern: '\\d{3}', |
There was a problem hiding this comment.
I think we need to wrap it with ^ and $
| }, | ||
| template: { | ||
| type: 'object', | ||
| additionalProperties: { type: ['string', 'null'] }, |
There was a problem hiding this comment.
Let's not distinguish null in schema.
We actually support null at every property (it's treated to be equivalent to undefined or not set at all)
Adding extra recognition is specific schema is not effective in any way
| items: { type: 'string' }, | ||
| }, | ||
| }, | ||
| oneOf: [{ required: ['origin'] }, { required: ['origins'] }], |
There was a problem hiding this comment.
I think we need to also introduce required: [], otherwise we make passing origin required (either via origin or origins option)
There was a problem hiding this comment.
It seems to me from the code that indeed one of them is required. We also had this manual validation that I removed which supports my theory.
There was a problem hiding this comment.
Indeed, it appears that if someone uses an object form, then not providing origin will fail.
Still that looks as apparent design bug, as by supporting cors: true we indicate that all properties have defaults. Intuitively object form should fall into origin: '*' if one was not set (it's e.g. the case in HTTP API, which has similar cors setting)
Anyway this is side issue (not a concern of this PR) so yes let's leave it as required.
medikoo
left a comment
There was a problem hiding this comment.
Thank you @thewizarodofoz it's great to have that in! Thanks for all the patience, that was tricky, as we learned this event properties set is large as has a lot quirks
Addresses: #8018
This PR is getting pretty big, so I think it would be wise to limit it to the http event schema (and do the provider schema in another PR). @medikoo WDYT?