I don't understand why you have a problem. make quality runs fine for me on master (and the CI is also happy).

Edit: Not saying that this is useless, I just want to understand why it fails for you and not for me :-)

Something is wrong on your side and CI.

check_repo.py actually did its job correctly and reported problems that this PR fixes. Have a look at the fix and tell me if that checker shouldn't have caught it.

Specifically:

• test_modeling_flax_bert.py and test_modeling_flax_roberta.py don't run common tests, and yet they weren't added to the ignore list
• there is no tests/test_modeling_flax_utils.py so modeling_flax_utils has to be on the other ignore list

I have no idea why you and CI don't reproduce these failures.

I'm in the middle of something else right now, but will investigate once I'm done. I'm unsure of why the CI hasn't been more angry with the jax PR myself. But I'd like to understand why it's not failing for me and the CI before taking the fix if that makes sense.

Absolutely, @sgugger. I will try to poke and see if the script behaves differently for some reason. I will report back if I find the culprit.

It should be safe for you to merge this as the lack of it may impacts other devs, I posted all the details why it's correct in here #7914 (comment)

It's your call.

Found part of the culprit - my py37 env doesn't catch the problem, whereas py38 does - now need to figure out whether it's some package difference or python itself.

Oh, interesting! I'm indeed on Python 3.7.9

and so is CI

Using my mightly https://github.com/stas00/conda-tools/blob/master/conda-env-compare.pl - I should get to the root of it in a few minutes

I downgraded the the py38 env to py37 and it doesn't detect the problem anymore. Upgraded it back to py38 via conda and it fails now too! bummer - so it must have been some package. I need to dig more.

I'd check jax and flax since the new flax code depends on it.

I'm confused by your report: by "it fails now too!" do you mean you don't see the problem anymore?

Edit: Think I've found the issue. It's because the presence/absence of jax/flax will change what's in the init. And neither me nor the CI have it installed.

If my reasoning is correct, #7919 should be red. We can then add it to your fixes and merge all of this together.

Yes, this is the culprit.

If you pip install jax jaxlib flax you should be able to reproduce the problem.

So basically the validation script is as good as the preinstalled pre-requisites allow it to be, therefore to move forward to do proper testing we need to have a prerequisites set that contains all possible external packages used by the core library.

Perhaps we need to change setup.py to add:

extras["all"] = list all groups here

and have the check_code_quality CI job installing `pip install -e .[all].

But specifically for this issue #7919 will do the trick. I merged it here as you suggested.

I'm confused by your report: by "it fails now too!" do you mean you don't see the problem anymore?

Yeah, I was trying to figure out the difference and there were too many differences in installed modules, so I downgraded to py37, then upgraded to py38 and lost an environment that was good for the purpose of this issue. I eventually recovered it. I need to remember to back up conda envs before I try to mess with them :(