Add support for Environment Variables to argparse

Ideas
1

Hello,

I’d like to (re-)open a discussion about adding support for environment variables to the argparse module in Python’s stdlib.

Background:

Nowadays it’s common for software to take arguments and configuration options via command line. I had a long list of examples here, but apparently “new users can only put 2 links in a post.”

The Issue

Although accepting environment variables (as an alternative way for passing command line arguments) is a fairly common thing, Python does not provide an “integrated” way of doing so for apps/scripts written in Python. While every developer can build something for his/her app based on os.getenv() and similar, every app where this is needed also has to implement the logic for doing so. This involves precedence (command line arguments take precedence over arguments passed via environment variables), naming the environment variables (e.g. the environment variable MYAPP_FOO can be used as a replacement for the –-foo command line option), and documentation (the developer will have to document the names of all the environment variables).

The current situation goes against the

There should be one-- and preferably only one --obvious way to do it.

from the “zen of python” and leads to inconsistencies in how various projects developed in Python handle environment variables.

My Proposal:

In my opinion, it would be great if this could be handled by the argparse module. I would propose to add a new (optional) argument for ArgumentParsers, I’ll call it envvar_prefix.

It could be used like this:

parser = argparse.ArgumentParser(envvar_prefix="MYAPP_")

I imagine the semantics as follows:

  • if envvar_prefix is not set, the argument parser will not consider any environment variables. (This should make the change perfectly backwards compatible.)
  • If envvar_prefix is set (it would have to be set to a string), it should be possible to pass every argument added to the parser via environment variables (see below for exceptions). For example, an argument called foo would be taken from the environment variable called envvar_prefix + "FOO", optional arguments such as --bar would be taken from envvar_prefix + "BAR".

Some more details:

  • The name of the environment variable for an argument would be constructed as parser.envvar_prefix + argument.name.upper().
  • arguments that can get multiple values (where nargs is +, * or > 1) should not be taken from environment variables. (One could discuss about using comma-separated values, e.g. MYAPP_FOO=bar,baz, but I’d like to keep things simple for now.)
  • Precedence: When calling parser.parse_args(), command line arguments should take precedence over environment variables and environment variables should take precedence over the default value (if any) of the argument.
  • envvar_prefix could be set to the empty string. If so, the environment variable corresponding to an argument named foo would simply be called FOO.
  • The envvar_prefix should apply to all subparsers, too. (Maybe it should be possible to specify additional prefixes for subparsers, so that args would be taken from an environment variable called root_parser.envvar_prefix + subparser.envvar_prefix + argument_name?)
  • I don’t know (yet) if there would be problems handling non-ascii argument names. For now I assume the argument name would simply be converted to upper-case, e.g. an argument called caña would be taken from MYAPP_CAÑA. I’d have to look into what this would mean for languages like chinese and arabic where there are no upper-case characters, my assumption is that Python’s upper() method doesn’t change the characters.

I’d love to get some feedback on the idea above and getting such a feature into Python’s stdlib.

References:

Ways of using environment variables for argparse have been discussed before, I could find the following discussions:

3 Likes
2

argparse is complicated enough without adding this functionality to it. I think a better approach would be adding a layer on top of argparse or maybe optparse (or both) that adds this functionality. There are packages in PyPI that already do this, like ConfigArgParse, which also supports config files.

6 Likes