Skip to content

Remove SPHINX_PARALLEL feature flag and make it the default - #7307

Closed
humitos wants to merge 1 commit into
masterfrom
humitos/remove-parallel-feature-flag
Closed

humitos wants to merge 1 commit into
masterfrom
humitos/remove-parallel-feature-flag

Conversation

@humitos

@humitos humitos commented Jul 21, 2020

Copy link
Copy Markdown
Member

We have been testing SPHINX_PARALLEL feature flag for some weeks now and we saw
some improvement on build times.

However, all the projects need to

  • install readthedocs-sphinx-ext>=2.1.0 (can be forced with
    USE_SPHINX_RTD_EXT_LATEST flag for now)
  • do not install any extension that does not support parallel reading
  • build on build:large queue in community (build:default has only one CPU)

As we are using -j auto, enabling it when it's not available won't be an issue
and builds should keep working properly.

We have been testing `SPHINX_PARALLEL` feature flag for some weeks now and we saw
some improvement on build times.

However, all the projects need to

* install `readthedocs-sphinx-ext>=2.1.0` (can be forced with
`USE_SPHINX_RTD_EXT_LATEST` flag for now)
* do not install any extension that does not support parallel reading
* build on `build:large` queue in community (`build:default` has only one CPU)

As we are using `-j auto`, enabling it when it's not available won't be an issue
and builds should keep working properly.
@humitos
humitos requested a review from a team July 21, 2020 16:51

@ericholscher ericholscher left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should also set this historical_default_true for a while prior to fully removing it, letting us roll it back easier.

@stsewd

stsewd commented Aug 5, 2020

Copy link
Copy Markdown
Member

This one only had 31 projects, maybe we should do a bigger sample before testing with default true.

@humitos

humitos commented Aug 18, 2020 •

Copy link
Copy Markdown
Member Author

We should also set this historical_default_true for a while prior to fully removing

Done!

/remind me about this in 15 days

@humitos humitos added the Status: blocked Issue is blocked on another issue label Aug 19, 2020
@humitos

humitos commented Aug 19, 2020

Copy link
Copy Markdown
Member Author

I'm marking this as blocked because it causes issues on projects that have fail_on_warning: true in their config file. This happens because our current deployed readthedocs-sphinx-ext does not declare read parallel safe and that makes Sphinx to raise a warning. We do have another feature flag to force using the latest version that declares that. However, if the project has any other extension that does not declare read parallel safe, it will raise this warning and may make current working builds to fail.

This issue was reported at #7394

@ericholscher

Copy link
Copy Markdown
Member

This is true for any extension that isn't parallel safe, which is most of them from my experience. I don't think we can default this on, and it needs to be configurable.

@humitos

humitos commented Jul 1, 2021

Copy link
Copy Markdown
Member Author

Closing this PR for now because there is no safe way to make it always work by default.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Status: blocked Issue is blocked on another issue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants