Allow running spellcheck or building docs separately #10377
Conversation
|
cc: @caddac |
docs/build_docs.py
Outdated
There was a problem hiding this comment.
LGTM, only question is if we do need to rename the file at all? i.e. just have docs/build instead of docs/build_docs.py
There was a problem hiding this comment.
Yes. There are several reasons for it:
-
there was a problem when I switched to Docker-build only tests in CI Images are now pre-build and stored in registry #10368 . In .dockerignore we are ignoring anything with "build" name at any level as build artifact. Usually generated directories (including one generated by setup.py create directories which are named "build". I could probably filter it out, but I think generally naming anything just "build" is a dangerous one precisely for the reason that various tools might skip or ignore it or delete it.
-
Without the .py extension this file does not get recognized as python file by pre-commit. This is why I had to fix a number of pylint/flake/mypy inside it it because it was not verified before.
-
I think generally speaking running it via buid.py should be discouraged in favour of
./breeze build-doc. The problem is that you never know if you have all the recent extensions installed. For example if you'd do it after adding spellchecking it would have failed and you would have to figure out that you have to re-rerun 'pip install .[doc]' again. For people who are casual contributors and do it very rarely this might come as a surprise. Also they might not remember that they have a separate virtualenv where they should build the docs and they would loose time trying to find out what's going on. They also might have an even older version of the virtualenv that will not work well.
The./breeze build-doc - handles all the venv problems for you and will ask you "rebuild the image" in such case,
which makes it obvious what you should do.
Actually - it's even worse now @kaxil @mik-laj - I just checked that now. And the old instructions are wrong. pip install -e '.[doc]' does not work at all:
If you do it from the scratch you get the below error. So unless we know how to fix it, I am removing the old mechanism and replace it with breeze-only. I see no point keeping the instructions which do not work.
Module "airflow.providers.amazon.aws.example_dags.example_datasync_1" could not be loaded. Full source will not be available. "error importing 'airflow.providers.amazon.aws.example_dags.example_datasync_1' (exception was: ModuleNotFoundError("No module named 'boto3'",))"
[airflow.providers.amazon.aws.example_dags.example_datasync_1] Entry is false for
[airflow.providers.amazon.aws.example_dags.example_datasync_1] Entry is false for
[airflow.providers.amazon.aws.example_dags.example_datasync_1] Entry is false for
Module "airflow.providers.amazon.aws.example_dags.example_datasync_2" could not be loaded. Full source will not be available. "error importing 'airflow.providers.amazon.aws.example_dags.example_datasync_2' (exception was: ModuleNotFoundError("No module named 'boto3'",))"
[airflow.providers.amazon.aws.example_dags.example_datasync_2] Entry is false for
Module "airflow.providers.amazon.aws.example_dags.example_ecs_fargate" could not be loaded. Full source will not be available. "error importing 'airflow.providers.amazon.aws.example_dags.example_ecs_fargate' (exception was: ModuleNotFoundError("No module named 'boto3'",))"
Module "airflow.providers.amazon.aws.example_dags.example_emr_job_flow_automatic_steps" could not be loaded. Full source will not be available. "error importing 'airflow.providers.amazon.aws.example_dags.example_emr_job_flow_automatic_steps' (exception was: ModuleNotFoundError("No module named 'boto3'",))"
[airflow.providers.amazon.aws.example_dags.example_emr_job_flow_automatic_steps] Entry is false for
Module "airflow.providers.amazon.aws.example_dags.example_emr_job_flow_manual_steps" could not be loaded. Full source will not be available. "error importing 'airflow.providers.amazon.aws.example_dags.example_emr_job_flow_manual_steps' (exception was: ModuleNotFoundError("No module named 'boto3'",))"
Module "airflow.providers.amazon.aws.example_dags.example_google_api_to_s3_transfer_basic" could not be loaded. Full source will not be available. "error importing 'airflow.providers.amazon.aws.example_dags.example_google_api_to_s3_transfer_basic' (exception was: ModuleNotFoundError("No module named 'boto3'",))"
[airflow.providers.amazon.aws.example_dags.example_google_api_to_s3_transfer_basic] Entry is false for
Module "airflow.providers.amazon.aws.example_dags.example_google_api_to_s3_transfer_advanced" could not be loaded. Full source will not be available. "error importing 'airflow.providers.amazon.aws.example_dags.example_google_api_to_s3_transfer_advanced' (exception was: ModuleNotFoundError("No module named 'boto3'",))"
[airflow.providers.amazon.aws.example_dags.example_google_api_to_s3_transfer_advanced] Entry is false for
[airflow.providers.amazon.aws.example_dags.example_google_api_to_s3_transfer_advanced] Entry is false for
[airflow.providers.amazon.aws.example_dags.example_google_api_to_s3_transfer_advanced] Entry is false for
[airflow.providers.amazon.aws.example_dags.example_google_api_to_s3_transfer_advanced] Entry is false for
==================================================
There was a problem hiding this comment.
I pasted a wrong (previous output) currently You have a lot of problems like that ^^ unless you have the [al] extras installed . I really think breeze is the only option for contributors to build the docs properly.
kaxil
changed the title
potiuk
force-pushed
the
add-selective-docs-behaviour
branch
from
c70159b to
1ff8b38
Compare
This cleans up the document building process and replaces it with breeze-only. The original instructions with `pip install -e .[doc]` stopped working so there is no point keeping them. Extracted from apache#10368
potiuk
force-pushed
the
add-selective-docs-behaviour
branch
from
1ff8b38 to
36dff7a
Compare
…he#10377) This cleans up the document building process and replaces it with breeze-only. The original instructions with `pip install -e .[doc]` stopped working so there is no point keeping them. Extracted from apache#10368 (cherry picked from commit 9228bf2)
…he#10377) This cleans up the document building process and replaces it with breeze-only. The original instructions with `pip install -e .[doc]` stopped working so there is no point keeping them. Extracted from apache#10368 (cherry picked from commit 9228bf2)
potiuk
added
the
type:misc/internal
…he#10377) This cleans up the document building process and replaces it with breeze-only. The original instructions with `pip install -e .[doc]` stopped working so there is no point keeping them. Extracted from apache#10368 (cherry picked from commit 9228bf2)
2 participants
Extracted from #10368
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.