Skip to content
/ django Public

Refs #32365 -- Allowed use of non-pytz timezone implementations.#13877

Merged
carltongibson merged 1 commit intodjango:masterfrom
pganssle:support_zoneinfo
Jan 19, 2021
Merged

Refs #32365 -- Allowed use of non-pytz timezone implementations.#13877
carltongibson merged 1 commit intodjango:masterfrom
pganssle:support_zoneinfo

Conversation

@pganssle
Copy link
Contributor

@pganssle pganssle commented Jan 11, 2021

The current plan for switching Django over to use the zoneinfo module is a hard switch in Django 4.0, but in the meantime it would be prudent to make it at least possible to use zoneinfo with Django. I am not sure if this will be a perfectly smooth experience for the end user, but I've added zoneinfo-based tests alongside (nearly) all pytz-based tests that I could find, then fixed any resulting errors.

I know the deadline is very tight on this so I did not try to get too fancy with any formatting or the structures here so that we could get the review process started as soon as possible.

If this approach is acceptable, I think I still also need to add a few tests to:

  • tests/admin_widgets: unnecessary — pytz is not used for anything consequential
  • tests/admin_views: done
  • tests/db_functions/test_extract_trunc (truncating with ambiguous and imaginary times — this may need a logic change as well).

CC: @carltongibson @adamchainz

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: This idiom gets repeated in a bunch of places, not sure if there's a "test support" module or something you want me to put it in. I frequently bury this sort of thing in a _compat module somewhere.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we could introduce django.utils.zoneinfo like we did with simplejson back in the time.

We could start deprecating the module when we drop support for Python 3.8.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't want anything user-facing here, since this is only ever used in the tests. If there's a common test utility module somewhere in the test suite where I can add functions like, "Get this IANA key in all supported time zone providers" and "skip this test if zoneinfo isn't available", it would make the PR a bit cleaner, but probably a lot of this stuff will get refactored for 4.0 anyway, so I don't think it's critical.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right then I guess this import sequence is not too bad, it's already something we do in tests in a few other places when dealing with optional libraries.

@carltongibson
Copy link
Member

Hey @pganssle — thanks for this!

I've posted to the list about whether we can squeeze this in, if we can't get full review before Thursday.

Are the TODOs in your first comment still up to date?

@pganssle
Copy link
Contributor Author

pganssle commented Jan 12, 2021

@carltongibson I have updated the TODOs accordingly. I think that there are two remaining questions here:

  1. Currently, make_aware will ignore is_dst for non-pytz zones, because the semantics of fold and is_dst is not a perfect mapping. Since fold is set on the value parameter and has no default, there's no way to detect if someone has deliberately set fold and is_dst, and it's not clear which should take precedence.

    I could reasonably migrate the logic from pytz-deprecation-shim here, which is that is_dst is ignored when not specified (value's fold is used instead), and if explicitly specified, normal pytz logic applies (we'd need to swap out the AmbiguousTimeError and NonExistentTimeError for another exception type in the None case, obviously). In the end I think is_dst should be deprecated or removed or something in whatever version built-in zoneinfo support rolls out in, since the semantics aren't quite right.

  2. A related concern is what to do about this date truncation logic. This is similar to the other issue in that it comes down to how is_dst should work. The difference is that with make_aware, you are passing a datetime directly, and can thus set the fold. In this case the datetime is created for you behind the scenes, and you only have the is_dst lever. We can leave this as-is and have people "fix it in post" if they opt-in, or we can add the is_dst resolution logic directly into the TruncBase.convert_value code, or we can add the is_dst support to make_aware and the truncation logic will be inherited automatically.

That stuff is the only possible backwards-compatibility concern in this PR, since to the extent that it's possible to use make_aware and the truncation logic with non-pytz zones today, is_dst is already ignored. Making it sometimes raise exceptions could be a backwards incompatible change (though if we're taking something that was broken and unusable and making it usable maybe that's not a really incompatible change). One option to get around this would be to disable the is_dst=None behavior for non-pytz zones and add a new singleton like django.utils.timezone.ERROR_UNLESS_UNAMBIGUOUS (or accept the string 'error') that triggers the is_dst=None behavior even in non-pytz zones.

In any case, I would say that even without any changes to the is_dst logic, this PR is a win for DJango users, and we shouldn't let the perfect be the enemy of the good. I think any wonky DST logic that would come about from the lack of is_dst support for non-pytz zones could be fixed reasonably easily by end users anyway.

Copy link
Member

@adamchainz adamchainz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are some pytz references in docs/i18n/timezones.txt that should likely be updated to be about "your timezone provider"

@pauloxnet
Copy link
Member

It seems ok to me. I added only a small comment.

@pganssle
Copy link
Contributor Author

@adamchainz I've made some minor updates to the i18n documentation, but a lot of the references to pytz are very specific to pytz, and might require a more significant overhaul to show both ways of doing things.

I'd also say that this advice, while not so pytz-specific, is a bit of a problematic recommendation, as the tz project is pretty clear that the database keys that pytz.common_timezones and zoneinfo.available_timezones() expose are not really intended to be user-facing, but "how to create a timezone-picker" is a much more difficult and complicated topic to address.

I think for now it's probably best to just acknowledge that non-pytz zones are supported (and intended to be supported), and decide later on whether it's worth overhauling the docs to show both pytz and zoneinfo support side-by-side.

Copy link
Member

@aaugustin aaugustin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I don't have time to perform a full review. At a high level this looks OK to me. Thanks Paul & everyone else involved!

@carltongibson carltongibson changed the title Add support for non-pytz zones Refs #32365 -- Allowed use of non-pytz timezone implementations. Jan 19, 2021
@carltongibson carltongibson force-pushed the support_zoneinfo branch 2 times, most recently from 1f58f2f to 3faef6f Compare January 19, 2021 10:33
@carltongibson carltongibson merged commit 10d1261 into django:master Jan 19, 2021
@aaugustin
Copy link
Member

🎉

@adamchainz
Copy link
Member

@pganssle
Copy link
Contributor Author

Thanks so much for your help with this everyone. This was a great first experience contributing to Django ❤️

terencehonles added a commit to terencehonles/django-rest-framework that referenced this pull request Apr 7, 2021
adamchainz pushed a commit to encode/django-rest-framework that referenced this pull request Apr 16, 2021
After django/django#13877, Django no longer checks for `hasattr(timezone, 'localize')` and instead does an inheritance check.
sigvef pushed a commit to sigvef/django-rest-framework that referenced this pull request Dec 3, 2022
After django/django#13877, Django no longer checks for `hasattr(timezone, 'localize')` and instead does an inheritance check.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants