dotenv-linter icon indicating copy to clipboard operation
dotenv-linter copied to clipboard

Question: is there a way to filter certain comparisons?

Open MetalArend opened this issue 3 years ago • 24 comments

If you have .env and .env.example files, it makes sense to compare both files to have the same keys. If you have .env and .env.local, it is quite normal to not have all keys in the .env.local file, but the keys might still have to be the same (even when sometimes a .env.local file might have completely different keys as well).

Is there a way to tell the compare command about the second situation, to compare a local overwrite file to the keys in the actual .env file?

MetalArend avatar Aug 12 '22 21:08 MetalArend

Hi, just to clarify, could you provide a small example with the expected behavior?

DDtKey avatar Aug 13 '22 09:08 DDtKey

.env:

BAR=FOO
FOO=BAR

.env.local

FOO=BAR

This should be valid, as a .env.local file is only used to overwrite certain parameters, but will probably never have all the same variables as the .env file.

So, is there a way to run dotenv-linter compare .env .env.local, and make it pass?

MetalArend avatar Aug 13 '22 10:08 MetalArend

I was thinking it might be an option to make a skip parameter in the same way as the check or fix command? But afaik there are no rules for the comparison as of yet?

MetalArend avatar Aug 13 '22 10:08 MetalArend

So do you mean checking required params for overriding? 🤔 Something like inheritance-based comparisons

Not sure if it's comparator responsibility, it just returns differences between files, while we talk about interpretation of these differences. However it could be some extension-functionallity, of course

@dotenv-linter/core what do you think of it, guys?

DDtKey avatar Aug 13 '22 10:08 DDtKey

I don't mind, but I don't fully understand how it will work.

What will be a trigger for that feature? The .local extension or an additional flag? What will be the output of the program?

mgrachev avatar Aug 15 '22 12:08 mgrachev

If you would have --skip for the compare command, you could have rules like for example AllKeysShared or something like that? So you could run dotenv-linter compare --skip AllKeysShared? Does that make sense somehow?

MetalArend avatar Aug 16 '22 09:08 MetalArend

TBH, I still can't understand how it should works.

If we allow one of the files not to have keys, we need to understand which ones. Otherwise, what are we even comparing? 🙂

I thought you want something like this:

# .env

BAR=FOO
# dotenv-linter:inheritance required      <--- so we allow to have a separate group of required variables
FOO=BAR
# .local.env

FOO=BAR

And dotenv-linter compare --inheritance .env .local.env won't return errors (or even separate feature: dotenv-linter inheritance .env .local.env

DDtKey avatar Aug 16 '22 10:08 DDtKey

A .env.local file is a local overwrite of some of the .env values. But the .env cannot have extra inheritance comments, as there is no way of knowing which values a developer wants to overwrite, and a .env.local file is sometimes not even a requirement. But what we do know about .env.local files, is that the keys of the .env.local file need to be in the range of keys that are defined in the .env file or a .env.example file or .env.dist file. I would like to be able to check the .env.local file to not have keys that are not defined in another file, but without the neccesity to have all keys in the .env.local file.

If I would run dotenv-linter compare --skip "AllKeysOverwritten" .env .env.local, the .env.local should be completely compared to .env, but the latter may have certain keys not defined. It could work with an --inheritance flag or command, but then again the logic is completely moved to a flag or even another command, which might be less flexible. The awesomeness imo of the --skip flag as implemented in the check command, is that you can individually say which checks do not need to run. So instead of naming it "inheritance", define what the "inheritance" disables in the checks: "AllKeysOverwritten"/"AllKeysPresent".

What is the reasoning behind the separate command? Because the compare command can have 3 files as arguments, and does not add inheritance to them based on the order?

MetalArend avatar Aug 16 '22 11:08 MetalArend

I assumed you wanted to be able to check that all required variables have been overridden in case of inheritance. If we are only talking about the fact that all the .env.local keys are in the range of the .env - it looks reasonable.

I was personally confused by this:

(even when sometimes a .env.local file might have completely different keys as well).

If the local can contain keys that are not in the parent at all, then a mechanism for specifying variables that are required is needed. And in this case, it is already more than a "comparison" in the conceptual sense (and yes, it could have a different semantic for several args)

DDtKey avatar Aug 16 '22 11:08 DDtKey

If the local can contain keys that are not in the parent at all, then a mechanism for specifying variables that are required is needed.

Ah, indeed, sorry about the confusion. I considered it a good trade-off to have to mention a certain key in the .env to get through the comparison with dotenv-linter, as most of the times having a .env.local file with an unknown key is well worth documenting the situation for the setup that needs that unknown key.

MetalArend avatar Aug 16 '22 12:08 MetalArend

This issue is stale because it has been open 14 days with no activity. Remove stale label or comment or this will be closed in 5 days.

github-actions[bot] avatar Aug 31 '22 04:08 github-actions[bot]

Burnout bot is around :D

MetalArend avatar Aug 31 '22 21:08 MetalArend

I believe we need to formalize what we've discussed a bit in a more readable format. In this case, someone will take it faster.

To put it briefly: Add the ability to check the occurrence of the variables set X in Y for compare command (and not just the complete matching of keys).

So expected behavior: env-file X( e.g .env.local) must only contain keys from file Y (e.g .env), but some of them can be omitted

Valid example:

# .env
BAR=1
FOO=2
# .env.local
FOO=3

It make sense for overriding scenarios. The flag to enable this behavior still under discussion, some examples:

  • dotenv-linter compare --skip "AllKeysOverwritten" .env .env.local
  • dotenv-linter compare --override .env .env.local
  • your suggestion ?

@dotenv-linter/core

@MetalArend correct me if I'm wrong

DDtKey avatar Sep 01 '22 09:09 DDtKey

@DDtKey That sounds right :) Typical examples in the wild are .env and .env.local, where one may overwrite some of the keys, based on their own decision and situation. Keyword being may.

MetalArend avatar Sep 03 '22 05:09 MetalArend

I would think about the API of the command, because users can change the position of arguments or pass they more than need, e.g.:

$ dotenv-linter compare --override .env.local .env

$ dotenv-linter compare --override .env.local .env.test .env

$ dotenv-linter compare --override .env.local .env .env.test

mgrachev avatar Sep 14 '22 10:09 mgrachev

In those examples, I would think the .env.local belongs to the override flag, the other multiple arguments are the original env files? Makes sense :)

MetalArend avatar Sep 15 '22 23:09 MetalArend

This issue is stale because it has been open 14 days with no activity. Remove stale label or comment or this will be closed in 5 days.

github-actions[bot] avatar Sep 30 '22 04:09 github-actions[bot]

This issue was closed because it has been stalled for 5 days with no activity.

github-actions[bot] avatar Oct 06 '22 03:10 github-actions[bot]

I would add an additional command - compare-override instead of adding a --override flag to the existing command compare:

$ dotenv-linter compare-override .env .env.local

This will simplify the code, because I don't want to add many conditions to the existing command 🙂

@dotenv-linter/core What do you think?

mgrachev avatar Oct 23 '22 10:10 mgrachev

This issue is stale because it has been open 14 days with no activity. Remove stale label or comment or this will be closed in 5 days.

github-actions[bot] avatar Nov 07 '22 03:11 github-actions[bot]

This issue was closed because it has been stalled for 5 days with no activity.

github-actions[bot] avatar Nov 13 '22 03:11 github-actions[bot]

I think the approach here would be a cool way to implement this.

Here are some ideas for your consideration for annotations that could help various scenarios:

# .env.baseline

# dotenv-linter:inheritable
INHERITABLE_VALUE=foo

# dotenv-linter:inheritable-overridable
OVERRIDEABLE_INHERITABLE_VALUE=foo

# dotenv-linter:inheritable-must-override
MUST_BE_SPECIFIED_IN_CHILD=foo


# .env.dev

# lint emits warning for duplicate
INHERITABLE_VALUE=realthing

# lint emits no warning
OVERRIDEABLE_INHERITABLE_VALUE=realthing

# if this value isn't specified then we should get an error
MUST_BE_SPECIFIED_IN_CHILD=realthing

jblotus avatar Jan 09 '23 17:01 jblotus

To follow up with that - if the inheritance "rules" were implemented maybe dotenv-linter could dump out an .env file that combined the "base" file and the "child". This would allow flattening for analysis purposes or simplify deployment time, or maybe useful for debugging inheritance in .env files which can be tricky.

jblotus avatar Jan 09 '23 17:01 jblotus

seems to me there needs to be a merge option. read n files in, then output the merge of them all, with the later ones taking priority over the earlier ones. Not really 'lint' but more 'management'

pm100 avatar Mar 04 '24 01:03 pm100