Settings: lists or tuples?

Hello,

I’m about to introduce a new setting that is a sequence of things, TEMPLATES, and I have to decide whether it will be a tuple of a list.

Unfortunately Django is very inconsistent in the type of settings. In global_settings.py:

- 22 settings are tuples
- 10 are empty, 12 aren't
- 12 are referred to as “lists” in code comments, 1 as “tuple”
- 6 settings are lists
- all of them are empty

Even the tutorial is inconsistent. The first two settings described are:

- INSTALLED_APPS: example is a tuple
- TEMPLATE_DIRS: example is a list

It would be nice to standardize on tuples or lists — at least for new code if it’s undesirable to change the existing code.

This is purely a matter of consistency and aesthetics. Using a tuple or a list never makes any difference in the code.

While lists are less common than tuples currently, I prefer them for two reasons:

1) All these settings are sequences of similar things. Such values are best represented with lists, unless they have to be immutable, in which case a tuple can be used. (tuples are both “namedtuples without names” and “immutable lists” in Python.)

2) Lists aren’t prone to the “missing comma in single-item tuple” problem which bites beginners and experienced pythonistas alike. Django even has code to defend against this mistake for a handful of settings. Search for “tuple_settings” in the source.

Surprisingly, it seems that there isn’t a backwards compatibility problem with changing the type to lists, as the following pattern still works:

# myproject/settings.py
from django.conf.global_settings import FILE_UPLOAD_HANDLERS
FILE_UPLOAD_HANDLERS += ('myproject.uploadhandler.FooBarUploadHandler',)

Proof:

>>> foo = ['abc']
>>> foo += ('def',)
>>> foo
['abc', ‘def’]

I can’t think of another circumstance it which the type of the default value will make a difference.

So… can we normalize the code and satisfy my OCD? Or should I just stop caring and move on? ;-)

--
Aymeric.

I agree with your preference for lists, for the same two reasons.
Unfortunately `__iadd__` and `__add__` do not have the same behavior here:

>>> foo = ['abc']
>>> foo = foo + ('def',)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: can only concatenate list (not "tuple") to list

So I think there is a backwards-compatibility issue.

Personally, I would love to decide to just bite this bullet and
normalize to lists, but I think it would need to be marked as a
backwards-incompatibility in the release notes, and it would certainly
bite people.

Carl

Indeed.
We can do that. The error is easy to fix if it occurs. Technically
django.conf.global_settings is a private API.

But I feel guilty to break code just for consistency and aesthetics :-/

--
Aymeric.

This is clever, but on second thought I'm trying to figure out in what
scenario a backwards-compatibility would actually occur here. In the
context of a user's settings modules, global_settings plays no role at
all: you can't add to a global setting default in your settings.py
(unless you've imported global_settings yourself, which is certainly
making use of undocumented internals).

So it seems to me that the only scenario in which this would actually
cause a back-compat problem would be if someone is using a tuple
setting, which they are not setting themselves (and thus are using the
global default), and they are adding to it in their runtime code (not in
their settings file). For instance:

my_ctx_procs = settings.TEMPLATE_CONTEXT_PROCESSORS + ('another',)

The common case of appending to a setting in your settings.py (due to
use of some kind of split-settings) should not be impacted at all.

I guess this is still enough of a backwards-incompatibility that we
should have a deprecation process, but it won't bite nearly as many
people as I'd initially thought it would.

FWIW, I think the subclass-of-list approach is neat enough, I'd be in
favor of going ahead and doing this with that approach, if someone's
motivated to write the patch.

Carl

What would __iadd__ do in this subclass? Would it behave like tuple and
create a new DjangoList, or would it behave like list and extend the
original list?

I’m interested at taking a whack at it, though I can imagine somebody else
getting it done faster.

Ryan

For maximum backwards-compatibility during the deprecation period, if
it's called with a tuple, it should create and return a new tuple, I
think. That way the deprecation warning only gets fired once for a
particular setting, which is all that's needed, even if they go and add
to the resulting value several more times.

If it's called with a list, it should behave as a normal list does.

And before you get too far, you may want to wait and see if any other
core devs step in to tell me I'm crazy and there's no way we should have
a deprecation path just to switch some settings from tuples to lists.

(I'm only in favor of it myself because I think in practice the impact
will be small.)

Carl

Thanks for the warning, and your solution makes perfect sense to me.
I’m gonna go ahead and keep working on it for fun, but I understand
that we might find that it’s not worth the deprecation period, and
I’m OK with that.

Thinking about it a bit, I’m going to see if just watching for tuple
and returning a tuple, otherwise returning a native list might be
sufficient.

Ryan

Sounds good. In case anyone is interested, I made a pull request (not complete) with an implementation of a ‘ConfList’ that could detect operations with tuples. Scrapping it doesn’t bother me, it was a fun challenge. Perhaps someone else will think it’s interesting too.

https://github.com/django/django/pull/3752

Works for me!

Carl

Hi Andreas,

As Florian pointed out, you can't mutate a tuple, but you can mutate the settings object, so using a tuple or a frozendict won't buy you much.

Regarding the theoretical perspective, tuples aren't meant to be immutable lists. Tuples are for heterogenous collections of fixed size (e.g. GPS coordinates); whereas lists are for homogenous collections of variable size, which suits perfectly our settings use-case. Of course Python being Python you can use them however you like but let's not talk about theory in that case.

Cheers

--
Loïc
> To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/07fd597a-01d2-4f6c-845b-5587a19b3263%40googlegroups.com.

Presumably more than just my personal taste: https://docs.python.org/2/faq/design.html#why-are-there-separate-tuple-and-list-data-types.

--
Loïc

There is nothing in this anecdote that is specific to Django settings,
or even to Django. The appropriate lesson to be learned from it is
general to Python: no function that takes mutable objects as parameters
should ever mutate those objects (unless that side-effect is its
intended purpose, and that fact is clearly advertised in its
contract/documentation/docstring). Hopefully the second colleague in
your anecdote learned that lesson.

I don't see a reason here to prefer tuples over lists in settings.

Carl

Hi Andreas,

This is probably obvious, but I thought I’d mention it just in case — you can keep
using tuples in your projects. We’re just talking about standardizing the docs and
the default values.

I understand your arguments and I can see how your experience led you there.
However, like others, I believe that encouraging “almost immutable” values would
provide a false sense of security and cannot be a substitute to an understanding
of Python’s data model.

By “almost immutable” I refer to the fact that even if settings.FOO is immutable
you can still overwrite it, especially with += whose difference with append will be
lost on someone who doesn’t understand the data model.

--
Aymeric.

As explained in my previous email, tuples don’t help much. I’m against stating that using tuples is somehow safer.

--
Aymeric.

It is incorrect to say that the reason for lists is "beginner
friendliness". It may well make things friendlier for beginners, but
what it definitely helps with is for more advanced deployment
configuration and for modifying settings from base or default settings
files.

Secondly, settings are *only* altered at run time. The module pointed
at by DJANGO_SETTINGS_MODULE is imported, its code is run, and the
settings object is produced. From this point on, yes, settings should
not be modified, but that tells nothing about how they can be modified
within the settings module during runtime but before being frozen in
the settings object.

From my POV therefore, it is doubly wrong to have tuples and say this
is because the values are meant to be immutable, because it does not
reflect reality.

For instance:

from project.default_settings import *
DEBUG=True
INSTALLED_APPS.append('django_debug_toolbar')
LOGGING['loggers']['homepage'] = { 'handlers': 'logfile', 'loglevel': DEBUG }

All of this modification is perfectly valid, and this is why (imo) the
default should be list instead of tuples.

Cheers

Tom