The return of Python dictionary "addition"

Back in March, we looked at a discussion and Python Enhancement Proposal (PEP) for a new dictionary "addition" operator for Python. The discussion back then was lively and voluminous, but the PEP needed some updates and enhancements in order to proceed. That work has now been done and a post about the revised PEP to the python-ideas mailing list has set off another mega-thread.

PEP 584 ("Add + and += operators to the built-in dict class") has gotten a fair amount bigger, even though it has lost the idea of dictionary "subtraction", which never gained significant backing the last time. It also has two authors now, with Brandt Bucher joining Steven D'Aprano, who wrote the original PEP. The basic idea is fairly straightforward; two dictionaries can be joined using the "+" operator or one dictionary can be updated in place with another's contents using "+=". From the PEP:

    >>> d = {'spam': 1, 'eggs': 2, 'cheese': 3}
    >>> e = {'cheese': 'cheddar', 'aardvark': 'Ethel'}
    >>> d + e
    {'spam': 1, 'eggs': 2, 'cheese': 'cheddar', 'aardvark': 'Ethel'}
    >>> e + d
    {'cheese': 3, 'aardvark': 'Ethel', 'spam': 1, 'eggs': 2}
    
    >>> d += e
    >>> d
    {'spam': 1, 'eggs': 2, 'cheese': 'cheddar', 'aardvark': 'Ethel'}

As can be seen, it is effectively an "update" operation (similar to using the .update() method) where the last value for a particular key "wins". That is why "cheese" is "cheddar" for d + e, but it is 3 for e + d. The example also shows that the operation is not commutative, which bothered some commenters even though there are already several such "arithmetic" operators that are not commutative; list "addition" using "+" isn't either, for example.

There were some objections to removing subtraction, some +1 and -1 responses, and others along those lines, but the biggest chunk of the thread was taken up by the question of how to "spell" the operator. The question seems to boil down to whether to use "|" instead of "+"; that was also part of the discussion back in March and is mentioned in the PEP as well. The operation is seen by some as being analogous to the set union operation, which uses "|".

Richard Musil kicked off a big sub-thread by making the argument for the set-union usage, though he suggested an entirely new operator ("|<") for it. He is concerned about the ambiguity of the + operator in Python and that choosing something completely new will ensure that users do not guess incorrectly about what it does. Chris Angelico did not see things that way, however:

But Paul Moore is unsure that there is any real need for a new dictionary addition operator:

He said that he has never needed that kind of operator and suggested that someone do a survey of real-world Python code to see if it would be improved using the new operator, though he did admit to not following the debate closely. It turns out that a big chunk of the PEP is taken up by examples of how the new operator might be used, taking examples from third-party code (including SymPy and Sphinx). Moore was not entirely impressed with them, however, saying that only four out of the roughly 20 examples were improved with the switch, though another few were arguable.

Andrew Barnert thought that Moore's observation actually made a good argument in favor of the proposal; if those who are not in favor of the proposal think that roughly a quarter of the examples are an improvement using it, that's a pretty strong vote in its favor. Beyond that, though, he thinks the need for + (which he calls "copying update") makes for a compelling case, more so than just for the += operator ("mutating update"):

With the "{**a, **b}" example, he is referring to using the dictionary unpacking operator, "**", which is specified in PEP 448 ("Additional Unpacking Generalizations"), to do the update operation. While that "works", it suffers from the drawbacks he mentions; it is also a fairly universally disliked language idiom. Most are fine with the "**" operator itself, but using it in that way is considered rather non-obvious and is quite unpopular.

D'Aprano pointed out that adding two dictionaries using + has come up frequently over the years, seemingly independently; "to many people, the use of + for this purpose is obvious and self-explanatory". The thread continued with some arguing for each spelling of the operator; in some sense, the arguments often came down to "taste". There were also some more exotic ideas (spellings other than + or |, providing a "did you mean ... ?" kind of error for + to lead users to |, and so on), but Guido van Rossum said he is "not crazy" about the "did you mean ... ?" idea; he indicated that he sees the field as already having been narrowed down:

Beyond that, his preference would be to use |, but he is not completely opposed to +:

While the discussion went on at length, no real consensus was reached. As is always the case in the Python world, though, it seems, the discussion was never heated or even contentious really; in the end it comes down to personal preferences. As D'Aprano put it, even if the PEP "fails", it will have succeeded at some level:

One would guess that the discussion will move from python-ideas to python-dev before too long and then likely to the steering council for some kind of decision. We know how one member of the council (Van Rossum) is leaning at this point, but we'll have to wait and see how the rest of that group feels as none have been active in the discussion. It seems like a reasonable "addition" to the language, however spelled, though using + seems more likely to head off newbie queries. Lists and dictionaries are much more integral to Python; those who are new to the language will probably see list "addition" well before they ever meet sets.

>>> a = collections.Counter({'a': 1})
>>> b = collections.Counter({'a': 2})
>>> a + b
Counter({'a': 3})