Skip to content

Add jupyter and toml file types#44

Merged
charliermarsh merged 2 commits intoastral-sh:mainfrom
hamdanal:jupyter
Jul 9, 2023
Merged

Add jupyter and toml file types#44
charliermarsh merged 2 commits intoastral-sh:mainfrom
hamdanal:jupyter

Conversation

@hamdanal
Copy link
Copy Markdown
Contributor

@hamdanal hamdanal commented Jul 7, 2023

@charliermarsh
Copy link
Copy Markdown
Member

Thanks! Do you mind adding toml too?

Comment thread .github/workflows/main.yml Outdated
@charliermarsh charliermarsh changed the title Add jupyter file type Add jupyter and toml file types Jul 9, 2023
@charliermarsh charliermarsh merged commit 89791c7 into astral-sh:main Jul 9, 2023
@charliermarsh
Copy link
Copy Markdown
Member

Thanks!

@hamdanal
Copy link
Copy Markdown
Contributor Author

Hi @charliermarsh, does ruff lint toml files? Or did you ask for toml to be added so that ruff has access to the configuration in pyproject.toml?

I tried this with one of my projects

- repo: https://github.com/astral-sh/ruff-pre-commit
  # Ruff version.
  rev: v0.0.277
  hooks:
    - id: ruff
      types_or: [python, pyi, jupyter, toml]
      args: [--fix, --exit-non-zero-on-fix]

And ruff started failing on the poetry.lock file (which is recognized by pre-commit as a toml file). It thinks this file is a python file and fails with a syntax error.
I am not sure toml should be added here.

Thank you for ruff by the way, it is an amazing tool.

@hamdanal hamdanal deleted the jupyter branch July 10, 2023 17:58
@charliermarsh
Copy link
Copy Markdown
Member

Yeah, Ruff lints pyproject.toml files (but no other TOML files). Hmm, we may need to revert that for now since we won't have a reliable way to detect TOML files if they're not using the .toml suffix.

@hamdanal
Copy link
Copy Markdown
Contributor Author

I'll send a PR to revert it and see how can we add pyproject.toml only.

@charliermarsh
Copy link
Copy Markdown
Member

Can you refresh my memory -- if we revert both of these, what happens if someone adds this to their configuration? Will Ruff still end up linting Jupyter notebooks?

- repo: https://github.com/astral-sh/ruff-pre-commit
  # Ruff version.
  rev: v0.0.277
  hooks:
    - id: ruff
      types_or: [python, pyi, jupyter]
      args: [--fix, --exit-non-zero-on-fix]

@charliermarsh
Copy link
Copy Markdown
Member

We may actually want to revert Jupyter too for now -- it should be considered opt-in, not on-by-default.

@hamdanal
Copy link
Copy Markdown
Contributor Author

hamdanal commented Jul 10, 2023

Can you refresh my memory -- if we revert both of these, what happens if someone adds this to their configuration? Will Ruff still end up linting Jupyter notebooks?

- repo: https://github.com/astral-sh/ruff-pre-commit
  # Ruff version.
  rev: v0.0.277
  hooks:
    - id: ruff
      types_or: [python, pyi, jupyter]
      args: [--fix, --exit-non-zero-on-fix]

Yes, you can override types_or locally in the user's repository

@hamdanal
Copy link
Copy Markdown
Contributor Author

hamdanal commented Jul 10, 2023

We may actually want to revert Jupyter too for now -- it should be considered opt-in, not on-by-default.

What about another hook like what black does. This way jupyter stays opt-in but also users don't have to learn about low level pre-commit configuration to make it work with notebooks.

@charliermarsh
Copy link
Copy Markdown
Member

I think my preference for now would be to revert, and add an example to the docs that users can copy-paste to add Jupyter support.

@hamdanal
Copy link
Copy Markdown
Contributor Author

I think my preference for now would be to revert, and add an example to the docs that users can copy-paste to add Jupyter support.

Done in #45

charliermarsh pushed a commit that referenced this pull request Jul 10, 2023
@charliermarsh
Copy link
Copy Markdown
Member

Thank you, sorry for the churn!

charliermarsh added a commit to astral-sh/ruff that referenced this pull request Aug 28, 2023
## Summary

This PR adds a higher-level enum (`SourceType`) around `PySourceType` to
allow us to use the same detection path to handle TOML files. Right now,
we have ad hoc `is_pyproject_toml` checks littered around, and some
codepaths are omitting that logic altogether (like `add_noqa`). Instead,
we should always be required to check the source type and handle TOML
files as appropriate.

This PR will also help with our pre-commit capabilities. If we add
`toml` to pre-commit (to support `pyproject.toml`), pre-commit will
start to pass _other_ files to Ruff (along with `poetry.lock` and
`Pipfile` -- see
[identify](https://github.com/pre-commit/identify/blob/b59996304fea80025d32abc3ac8f7212a9e7380d/identify/extensions.py#L355)).
By detecting those files and handling those cases, we avoid attempting
to parse them as Python files, which would lead to pre-commit errors.
(We tried to add `toml` to pre-commit here
(astral-sh/ruff-pre-commit#44), but had to
revert here (astral-sh/ruff-pre-commit#45) as it
led to the pre-commit hook attempting to parse `poetry.lock` files as
Python files.)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

types_or needs expanding?

2 participants