Skip to content

python: several typing and style improvements#16378

Merged
wing328 merged 4 commits intoOpenAPITools:masterfrom
lettuce-financial:typing-and-style-improvements
Sep 1, 2023
Merged

python: several typing and style improvements#16378
wing328 merged 4 commits intoOpenAPITools:masterfrom
lettuce-financial:typing-and-style-improvements

Conversation

@jessemyers-lettuce
Copy link
Copy Markdown
Contributor

The generated (python) code fails with several standard validation tools, including flake8, mypy, and autoflake. While fixing every possible violation -- especially wrto typing -- woudl be a project, some of the changes are fairly easy.

  • The autoflake tool picks up on unused imports. These can just be removed.

  • The mypy tool picks up on numerious typing violations, especially if set to its strictest mode. As a starting point, all functions ought to annotate a return type, including constructors, even if the return type is None because otherwise the functions are omitted from type checking and it's impossible to make incremental progress towards adding types.

  • The flake8 tool mostly finds whitespace and line-length issues; while line-length standards very, the source already includes several flake8 ignores, so it seems safe to add a few more.

Comment thread modules/openapi-generator/src/main/resources/python/api.mustache Outdated
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use of (object) as a base class has not been recommended for a long time.

@jessemyers-lettuce jessemyers-lettuce force-pushed the typing-and-style-improvements branch from d16c6d7 to d6af4ce Compare August 22, 2023 16:30
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The return type of -> None is necessary for the type checker to see a newly constructed instances as having a type. Otherwise, we end up with everything being any-typed.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This alignment avoids line-length issues (using black's line length configuration).

Comment on lines 78 to 79
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some formatters (e.g. black) will change:

raise ValueError("long string")  # noqa: E501

to:

raise ValueError(
    "long string"
)  # noqa: E501

thereby moving the noqa comment to the wrong location; declaring the message on its own line avoids this problem.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

flake8 does not like trailing spaces

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was just bad Python (and flake8 notices this)

@jessemyers-lettuce jessemyers-lettuce force-pushed the typing-and-style-improvements branch from d6af4ce to 77534ca Compare August 22, 2023 16:39
The generated (python) code fails with several standard validation tools,
including `flake8`, `mypy`, and `autoflake`. While fixing every possible
violation -- especially wrto typing -- woudl be a project, some of the
changes are fairly easy.

 - The `autoflake` tool picks up on unused imports. These can just be removed.

 - The `mypy` tool picks up on numerious typing violations, especially if set
   to its strictest mode. As a starting point, all functions ought to annotate
   a return type, including constructors, even if the return type is `None`
   because otherwise the functions are omitted from type checking and it's
   impossible to make incremental progress towards adding types.

 - The `flake8` tool mostly finds whitespace and line-length issues; while
   line-length standards very, the source already includes several flake8
   ignores, so it seems safe to add a few more.
@jessemyers-lettuce jessemyers-lettuce force-pushed the typing-and-style-improvements branch from 77534ca to 6e25048 Compare August 22, 2023 16:41
Comment on lines -7 to -9
import {{packageName}}
from {{apiPackage}}.{{classFilename}} import {{classname}} # noqa: E501
from {{packageName}}.rest import ApiException
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not clear what the right behavior is for a test case that is meant to be edited by hand. On the one hand, adding imports for likely usages is helpful; on the other hand, it's not great to generate code that doesn't pass the linter because it includes unnecessary imports.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about commenting out these imports instead so that users can easily uncomment these imports if needed?

def setUp(self):
self.api = {{apiPackage}}.{{classFilename}}.{{classname}}() # noqa: E501
def setUp(self) -> None:
self.api = {{classname}}() # noqa: E501
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The import block was importing both the package and the class; we only need one of these. It's more common in Python to use from foo import Bar style so that Bar() can be called without qualification.

Copy link
Copy Markdown
Contributor

@anis-campos anis-campos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, and I recently did the same in #16375 for python-flask.

Maybe we should start an issue that lists all the syntax and formatting issues of python projects, most likely, they will apply to most of them

@wing328
Copy link
Copy Markdown
Member

wing328 commented Aug 23, 2023

@jessemyers-lettuce thanks for the PR.

Maybe we should start a Issue that lists all the syntax and formatting issues of python projects, most likely, they will apply to most of them

@anis-campos yes please. Always good to have a ticket (issue) for tracking.

@wing328
Copy link
Copy Markdown
Member

wing328 commented Aug 23, 2023

@jessemyers-lettuce can you please follow step 3 in the PR checklist to update the samples so that the CI can verify the change?

https://github.com/OpenAPITools/openapi-generator/actions/runs/5941651979/job/16131743768?pr=16378

@wing328 wing328 added Client: Python Enhancement: Code format Enhancement: Code Cleanup General refactoring, removal of deprecated things, commenting, etc. labels Aug 23, 2023
@wing328 wing328 added this to the 7.0.0 milestone Aug 23, 2023
@jessemyers-lettuce
Copy link
Copy Markdown
Contributor Author

@wing328 I wanted to, but I don't have a working JVM in my setup; I invoke the generator via docker and work in other languages. If there's a way to run the build scripts through docker, I'm happy to help.

@jessemyers-lettuce
Copy link
Copy Markdown
Contributor Author

@wing328 I was able to get OpenJDK working and have now submitted.

There is an error that I do not understand. The removal of the unused Annotated import results in a NameError due to the samples using Annotated... despite there being no reference anywhere in modules/openapi-generator/src/main/resources/python to Annotated.

I'm not sure what to make of this; clearly some encapsulation boundary is being violated, but I'm reluctant to put in a great deal of effort if the scope of making improvements here requires developing deeper knowledge of the project; I'd hoped just to be able to improve the templates and move on. Please advise.

@wing328
Copy link
Copy Markdown
Member

wing328 commented Sep 1, 2023

FYI @taxpon (2017/07) @frol (2017/07) @mbohlool (2017/07) @cbornet (2017/09) @kenjones-cisco (2017/11) @tomplus (2018/10) @arun-nalla (2019/11) @krjakbrjak (2023/02)

@jessemyers-lettuce jessemyers-lettuce force-pushed the typing-and-style-improvements branch from e604a57 to 29e4f89 Compare September 1, 2023 15:40
@wing328
Copy link
Copy Markdown
Member

wing328 commented Sep 1, 2023

@jessemyers-lettuce
Copy link
Copy Markdown
Contributor Author

@wing328, I did, but I'm not seeing any changes when I do.

@wing328
Copy link
Copy Markdown
Member

wing328 commented Sep 1, 2023

python tests with updated samples passed via #16477. I'll merge this one and update the samples after.

Thanks again for the PR

@wing328 wing328 merged commit 40731ed into OpenAPITools:master Sep 1, 2023
@jessemyers-lettuce jessemyers-lettuce deleted the typing-and-style-improvements branch September 1, 2023 17:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Client: Python Enhancement: Code Cleanup General refactoring, removal of deprecated things, commenting, etc. Enhancement: Code format

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants