chore(mypy): add mypy support to makefile by michaelbrewer · Pull Request #508 · aws-powertools/powertools-lambda-python

michaelbrewer · 2021-07-06T04:09:43Z

Issue #, if available:

Official support for MyPy powertools-lambda#3

Description of changes:

Add mypy as a dev dependency
Add mypy as task in the Makefile
Put in some initial fixes (mostly just ignore untyped imports)

Example usage

$ make mypy 
poetry run mypy aws_lambda_powertools
aws_lambda_powertools/tracing/extensions.py:11: error: Skipping analyzing "aws_xray_sdk.ext.aiohttp.client": found module but no type hints or library stubs
aws_lambda_powertools/tracing/extensions.py:11: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
aws_lambda_powertools/tracing/base.py:9: error: Argument 1 to "contextmanager" has incompatible type "Callable[[BaseProvider, Any, KwArg(Any)], ContextManager[Any]]"; expected "Callable[..., Iterator[<nothing>]]"
aws_lambda_powertools/tracing/base.py:22: error: Argument 1 to "contextmanager" has incompatible type "Callable[[BaseProvider, Any, KwArg(Any)], AsyncContextManager[Any]]"; expected "Callable[..., Iterator[<nothing>]]"
aws_lambda_powertools/utilities/parameters/base.py:96: error: Incompatible types in assignment (expression has type "Union[Dict[Any, Any], bytes, None]", variable has type "str")
aws_lambda_powertools/utilities/parameters/base.py:154: error: Incompatible types in assignment (expression has type "Dict[str, str]", variable has type "Dict[str, Union[str, bytes, Dict[Any, Any], None]]")
aws_lambda_powertools/utilities/parameters/base.py:154: note: "Dict" is invariant -- see https://mypy.readthedocs.io/en/stable/common_issues.html#variance
aws_lambda_powertools/utilities/parameters/base.py:154: note: Consider using "Mapping" instead, which is covariant in the value type
aws_lambda_powertools/utilities/parameters/base.py:160: error: Incompatible types in assignment (expression has type "str", variable has type "Tuple[str, Optional[str]]")
aws_lambda_powertools/utilities/parameters/base.py:161: error: Argument 1 to "get_transform_method" has incompatible type "Tuple[str, Optional[str]]"; expected "str"
aws_lambda_powertools/utilities/parameters/base.py:165: error: Invalid index type "Tuple[str, Optional[str]]" for "Dict[str, Union[str, bytes, Dict[Any, Any], None]]"; expected type "str"
aws_lambda_powertools/utilities/parameters/base.py:165: error: Argument 1 to "transform_value" has incompatible type "Union[str, bytes, Dict[Any, Any], None]"; expected "str"
aws_lambda_powertools/utilities/parameters/base.py:169: error: Incompatible return value type (got "Dict[str, Union[str, bytes, Dict[Any, Any], None]]", expected "Union[Dict[str, str], Dict[str, Dict[Any, Any]], Dict[str, bytes]]")
aws_lambda_powertools/tracing/tracer.py:19: error: "LazyLoader" has no attribute "core"
aws_lambda_powertools/tracing/tracer.py:509: error: "Callable[..., Any]" has no attribute "__wrapped__"
aws_lambda_powertools/logging/formatter.py:109: error: Cannot assign to a method
aws_lambda_powertools/logging/formatter.py:109: error: Invalid self argument "LambdaPowertoolsFormatter" to attribute function "converter" with type "Callable[[Optional[float]], struct_time]"
aws_lambda_powertools/logging/formatter.py:131: error: Invalid self argument "LambdaPowertoolsFormatter" to attribute function "converter" with type "Callable[[Optional[float]], struct_time]"
aws_lambda_powertools/logging/formatter.py:131: error: Too many arguments
aws_lambda_powertools/logging/formatter.py:184: error: Argument 1 has incompatible type "str"; expected "Dict[Any, Any]"
aws_lambda_powertools/logging/formatter.py:204: error: Item "None" of "Optional[Type[BaseException]]" has no attribute "__name__"
aws_lambda_powertools/utilities/parameters/ssm.py:87: error: Signature of "get" incompatible with supertype "BaseProvider"
aws_lambda_powertools/utilities/data_classes/sqs_event.py:78: error: Return type "Optional[SQSMessageAttribute]" of "__getitem__" incompatible with return type "SQSMessageAttribute" in supertype "dict"
aws_lambda_powertools/utilities/data_classes/sqs_event.py:78: error: Return type "Optional[SQSMessageAttribute]" of "__getitem__" incompatible with return type "SQSMessageAttribute" in supertype "Mapping"
aws_lambda_powertools/utilities/data_classes/sqs_event.py:80: error: Argument 1 to "SQSMessageAttribute" has incompatible type "SQSMessageAttribute"; expected "Dict[str, Any]"
aws_lambda_powertools/logging/logger.py:355: error: Item "None" of "Optional[PowertoolsFormatter]" has no attribute "remove_keys"
aws_lambda_powertools/utilities/idempotency/persistence/base.py:104: error: Argument 1 to "loads" has incompatible type "Optional[str]"; expected "Union[str, bytes]"
aws_lambda_powertools/utilities/idempotency/persistence/base.py:286: error: Unsupported target for indexed assignment ("Optional[LRUDict]")
aws_lambda_powertools/utilities/idempotency/persistence/base.py:291: error: Item "None" of "Optional[LRUDict]" has no attribute "get"
aws_lambda_powertools/utilities/idempotency/persistence/base.py:301: error: Unsupported right operand type for in ("Optional[LRUDict]")
aws_lambda_powertools/utilities/idempotency/persistence/base.py:302: error: Item "None" of "Optional[LRUDict]" has no attribute "__delitem__"
aws_lambda_powertools/middleware_factory/factory.py:121: error: Item "None" of "Optional[Callable[..., Any]]" has no attribute "__qualname__"
Found 29 errors in 10 files (checked 108 source files)
make: *** [mypy] Error 1

Checklist

Meet tenets criteria
Update tests
Update docs
PR title follows conventional commit semantics

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

codecov-commenter · 2021-07-06T04:11:30Z

Codecov Report

Merging #508 (eff22cb) into develop (43b828e) will not change coverage.
The diff coverage is 100.00%.

@@           Coverage Diff            @@
##           develop     #508   +/-   ##
========================================
  Coverage    99.90%   99.90%           
========================================
  Files          107      107           
  Lines         4280     4280           
  Branches       212      212           
========================================
  Hits          4276     4276           
  Misses           1        1           
  Partials         3        3

Impacted Files	Coverage Δ
aws_lambda_powertools/logging/logger.py	`100.00% <100.00%> (ø)`
aws_lambda_powertools/utilities/validation/base.py	`100.00% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 43b828e...eff22cb. Read the comment docs.

heitorlessa

Let's create a mypy.ini file to best ignore untyped libraries among other features we can enable like a pydantic plugin.

Config: https://mypy.readthedocs.io/en/stable/config_file.html

Example that we can start with, errors will become easier to read too:

[mypy]
warn_return_any=False
warn_unused_configs=True
no_implicit_optional=True
warn_redundant_casts=True
warn_unused_ignores=True
pretty = True
show_column_numbers = True
show_error_codes = True
show_error_context = True

[mypy-jmespath]
ignore_missing_imports=True

michaelbrewer · 2021-07-10T16:15:28Z

@heitorlessa no_implicit_optional=True seems a bit strict to me.

michaelbrewer · 2021-07-10T16:26:42Z

@heitorlessa no_implicit_optional=True seems a bit strict to me.

Before this was an implicit optional

def foo(msg: str = None):
   ...

now it needs to be

from typing import Optional


def foo(msg: Optional[str] = None):
   ...

heitorlessa · 2021-07-12T14:13:29Z

@heitorlessa no_implicit_optional=True seems a bit strict to me.

Before this was an implicit optional
def foo(msg: str = None):
   ...
now it needs to be
from typing import Optional


def foo(msg: Optional[str] = None):
   ...

This is correct actually. It's the issue that @Nr18 was having too. Challenge with Python is the mutable defaults

* develop: chore(deps): bump boto3 from 1.18.0 to 1.18.1 (aws-powertools#528) fix(tracer): mypy generic to preserve decorated method signature (aws-powertools#529) fix(parser): Make ApiGateway version, authorizer fields optional (aws-powertools#532) fix(mypy): fixes to resolve no implicit optional errors (aws-powertools#521) chore(deps): bump boto3 from 1.17.110 to 1.18.0 (aws-powertools#527) feat(feat-toggle): New simple feature toggles rule engine (WIP) (aws-powertools#494) chore(deps-dev): bump mkdocs-material from 7.1.9 to 7.1.10 (aws-powertools#522) chore(deps): bump boto3 from 1.17.102 to 1.17.110 (aws-powertools#523) chore(deps-dev): bump isort from 5.9.1 to 5.9.2 (aws-powertools#514) feat(mypy): add mypy support to makefile (aws-powertools#508) feat(api-gateway): add debug mode (aws-powertools#507)

feat(mypy): add mypy support

f10cc4e

boring-cyborg bot added area/logger dependencies Pull requests that update a dependency file internal Maintenance changes labels Jul 6, 2021

pull-request-size bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jul 6, 2021

Merge branch 'develop' into feat-makefile-mypy

0b97dd8

heitorlessa suggested changes Jul 9, 2021

View reviewed changes

feat(mypy): add mypy.init

eff22cb

pull-request-size bot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jul 10, 2021

heitorlessa approved these changes Jul 12, 2021

View reviewed changes

heitorlessa merged commit 27c82d8 into aws-powertools:develop Jul 12, 2021

heitorlessa changed the title ~~feat(mypy): add mypy support to makefile~~ chore(mypy): add mypy support to makefile Jul 12, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

chore(mypy): add mypy support to makefile#508

chore(mypy): add mypy support to makefile#508
heitorlessa merged 3 commits intoaws-powertools:developfrom
gyft:feat-makefile-mypy

michaelbrewer commented Jul 6, 2021 •

edited

Loading

Uh oh!

codecov-commenter commented Jul 6, 2021 •

edited

Loading

Uh oh!

heitorlessa left a comment

Uh oh!

michaelbrewer commented Jul 10, 2021

Uh oh!

michaelbrewer commented Jul 10, 2021

Uh oh!

heitorlessa commented Jul 12, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Comments

Conversation

michaelbrewer commented Jul 6, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of changes:

Example usage

Uh oh!

codecov-commenter commented Jul 6, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

heitorlessa left a comment

Choose a reason for hiding this comment

Uh oh!

michaelbrewer commented Jul 10, 2021

Uh oh!

michaelbrewer commented Jul 10, 2021

Uh oh!

heitorlessa commented Jul 12, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

michaelbrewer commented Jul 6, 2021 •

edited

Loading

codecov-commenter commented Jul 6, 2021 •

edited

Loading