Backport zoneinfo logic into dateutil.tz.tzfile by pganssle · Pull Request #1130 · dateutil/dateutil

pganssle · 2021-05-20T13:23:01Z

Summary of changes

This moves tzfile into its own submodule containing what is effectively a backport of the pure python implementation of zoneinfo.ZoneInfo, but with the semantics of a dateutil.tz.tzfile object.

The major differences from zoneinfo are:

All caching logic is implemented in tz.gettz rather than in tz.tzfile.
tzfile objects are pickled by value, not by reference to the IANA key.
tzfile has equality-by-value semantics, whereas zoneinfo has equality-by-identity semantics.

This does change the internal implementation details of the class. It also fixes at least two bugs:

It adds support for V2 and V3 files, which also solves the 2038 problem (and the "slim zoneinfo" problem).
Prior to this change, dateutil did not use the fold attribute during gaps (only during folds), in violation of PEP 495. This fixes that, but that does change the semantics of imaginary times a bit (and thus may affect some mechanisms for detecting imaginary times).

This is mostly a code dump with compatibility adjustments. It's quite possible that we'll want to refactor, particularly with respect to the way POSIX strings are handled. Theoretically tzfile could fall back to a tz.tzstr or some other tz.tzrange, or we could refactor tzstr and tzrange in terms of _CalendarOffset, _DayOffset and _TZStr (or both, by refactoring the internal logic of tzstr and/or tzrange).

Closes #462, #1059

Open Tasks:

Pull Request Checklist

Changes have tests
Authors have been added to AUTHORS.md
News fragment added in changelog.d. See CONTRIBUTING.md for details

mariocj89 · 2021-06-07T14:38:23Z

@pganssle do you want me to review this?

pganssle · 2021-06-09T14:09:02Z

@mariocj89 I still need to backport the tests, but you are welcome to review it. I expect in the long run we'll want to consolidate some of the logic around tzstr and tzrange and some of these custom rule representations built into the TZif parser (for the extended version 2 support that allows specifying a POSIX string), but I don't expect to overhaul anything else.

Also I believe some documentation changes will be necessary before we can merge this as well.

mariocj89 · 2021-06-09T15:25:17Z

OK, I'll take a look once all is ready.

Jorricks · 2022-07-25T20:27:17Z

So @mariocj89 I suppose this is really the only remaining blocker for dropping support for Python 2?

Dropping support for Python 2 would mean we could get into in-place typing info and all the new fun stuff right :)?

Given this has been open for ages, should I try to prioritize this MR, open a new MR and see if we can get somewhere within the next few months?

mariocj89 · 2022-07-26T11:16:07Z

@Jorricks this is indeed blocking most of the progress in the repo. If you have cycles to work on this we could drop Python 2 after we land and release this. @pganssle has been wanting to work on this for a long time but was not able to commit time to it.

kloczek · 2022-08-07T15:46:06Z

+1

Jorricks · 2022-08-11T19:35:45Z

Hi @mariocj89,
I'm attempting to start but I am missing some context. Could you help me out answering these questions :)?

How is this effort blocking the Python2 removal/deprecation effort?
If I understand correctly, the idea of this MR is to back port a copy of zoneinfo (that is only available in Python3.9+) into this application which is compatible with the current tzinfo class right?
Is the idea to implement this feature Python3.7+ or should it be Python2 compatible? Mostly asking because I'd like to add type hints already :).

mariocj89 · 2022-08-12T07:49:30Z

How is this effort blocking the Python2 removal/deprecation effort?

Without this change users in 2.7 will not get tz upgrades. This splits the tzdata from the package, allowing users to stay in an old version of the package but still get tzdata upgrades.

If I understand correctly, the idea of this MR is to back port a copy of zoneinfo (that is only available in Python3.9+) into this application which is compatible with the current tzinfo class right?

Correct

Is the idea to implement this feature Python3.7+ or should it be Python2 compatible? Mostly asking because I'd like to add type hints already :).

Needs to work for py2, so no type hints :).

Jorricks · 2022-08-12T09:28:26Z

How is this effort blocking the Python2 removal/deprecation effort?

Without this change users in 2.7 will not get tz upgrades. This splits the tzdata from the package, allowing users to stay in an old version of the package but still get tzdata upgrades.

If I understand correctly, the idea of this MR is to back port a copy of zoneinfo (that is only available in Python3.9+) into this application which is compatible with the current tzinfo class right?

Correct

Is the idea to implement this feature Python3.7+ or should it be Python2 compatible? Mostly asking because I'd like to add type hints already :).

Needs to work for py2, so no type hints :).

Now it completely makes sense! Thanks.

Jorricks · 2022-08-13T19:12:52Z

@mariocj89 to improve visibility on the progress & smoothen the process a little. Can we open a new branch containing this work so far?
We can just keep using this topic to keep track of everything that remains to be done so far I suppose.

I was diving a bit further (as I want to make sure I understand the full context before making changes) and was wondering the following:
During the building of the python-dateutil wheel, we package the dateutil-zoneinfo.tar.gz. Could(/Should?) this then also be replaced with a slim version of tzdata?

mariocj89 · 2022-08-15T11:33:52Z

@pganssle do you have some time to guide @Jorricks through this change please?

sfc-gh-izinkovsky · 2022-10-04T22:16:20Z

@pganssle are you still actively working on getting this change merged?

Jorricks · 2022-10-05T06:25:47Z

Unfortunately I don't have the time anymore to work on this and made zero progress anyway. So I don't think anyone is working on this.

The code is very complex and you are only doing this for python 2 users. Given that reached end of life years ago, I'm not to enthusiastic to work on this anyway.

This creates a fictional time zone Fictional/Liliput (a copy of Arctic/Longyearbyne) and puts it in a temporary directory that is used as the TZPATH, then we ensure that we can load the result.

This essentially backports the tests from the `zoneinfo` test suite.

We were building with fat because dateutil did not support the default (slim), but now it does, so we can switch to slim. It might make sense to run the tests with both fat and slim, but I am not convinced that it would be worth it.

pganssle force-pushed the new_tzfile branch from bd7a0b5 to 05fb944 Compare May 20, 2021 14:23

pganssle mentioned this pull request Aug 5, 2021

Add ability to get Olson/POSIX name of a timezone #76

Open

mariocj89 mentioned this pull request Feb 8, 2022

Python 2 deprecation schedule #653

Open

mariocj89 mentioned this pull request Mar 11, 2022

Move to src layout #1146

Merged

3 tasks

pganssle mentioned this pull request Mar 19, 2022

ENH: support zoneinfo tzinfos pandas-dev/pandas#46425

Merged

4 tasks

mariocj89 mentioned this pull request Apr 11, 2022

remove python2 #1202

Closed

mariocj89 mentioned this pull request May 5, 2022

Add support for Python 3.10 #1174

Merged

3 tasks

mcg1969 mentioned this pull request Oct 19, 2022

DNM: incorporate PR for special snowflake build AnacondaRecipes/python-dateutil-feedstock#3

Open

pganssle force-pushed the new_tzfile branch 6 times, most recently from 24c8f99 to 4d4bafa Compare December 3, 2022 20:33

pganssle force-pushed the new_tzfile branch 2 times, most recently from 06dd279 to e362489 Compare April 27, 2023 18:17

pganssle force-pushed the new_tzfile branch from f30137f to bc11478 Compare November 13, 2023 16:00

pganssle added 27 commits May 19, 2026 17:07

TEMP: Fast-fail

9be5019

Add test for TZPATH changes

92e7a62

This creates a fictional time zone Fictional/Liliput (a copy of Arctic/Longyearbyne) and puts it in a temporary directory that is used as the TZPATH, then we ensure that we can load the result.

Fix available_iana_timezones when tzdata is missing

a2cf606

WIP: Start backporting tests

4f65445

Skip some tests on rearguard

5c664ac

Fixes related to sub-minute offsets

afb4ea9

Test fixes related to sub-minute offsets

dfe40b8

Tests for various gaps and folds

a2e2dba

Add tests for setting fold when converting from UTC

d0977de

Add tests for datetime.time objects

fa28e48

Add test zone construction function

c66e08c

Add a test for a zone with 1 transition

fdec70e

Add a test with one DST -> DST transition

febbfcd

Add a test for no TZSTR set

0fc51e1

Add test for zones with no transitions

1b07539

Add test that empty zones raise

0baf0f9

Add tests for very large transition values

5e9a30f

Add tests for fixed offset zones with phantom transitions

9d7f1f5

Refactor weirdzone tests

29855aa

Add tzstr tests

5698829

Add key to repr

edac7f3

Add tzpath fixture

87f6280

Improve testing for new tzfile implementation

06ab449

This essentially backports the tests from the `zoneinfo` test suite.

Put tzwin as lower priority than tzfile

98aebb1

Documentation for the new tzfile

fd55700

Add changelog entry for dateutil#1130

5597bfb

Test with normal tzdata build instead of fat

0944966

We were building with fat because dateutil did not support the default (slim), but now it does, so we can switch to slim. It might make sense to run the tests with both fat and slim, but I am not convinced that it would be worth it.

pganssle force-pushed the new_tzfile branch from 6f54099 to 0944966 Compare May 20, 2026 00:07

pganssle added 2 commits May 20, 2026 11:02

fixup! Improve testing for new tzfile implementation

f9a4ca2

fixup! Improve testing for new tzfile implementation

5b3e1d5

Uh oh!

Conversation

pganssle commented May 20, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary of changes

Pull Request Checklist

Uh oh!

mariocj89 commented Jun 7, 2021

Uh oh!

pganssle commented Jun 9, 2021

Uh oh!

mariocj89 commented Jun 9, 2021

Uh oh!

Jorricks commented Jul 25, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mariocj89 commented Jul 26, 2022

Uh oh!

kloczek commented Aug 7, 2022

Uh oh!

Jorricks commented Aug 11, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mariocj89 commented Aug 12, 2022

Uh oh!

Jorricks commented Aug 12, 2022

Uh oh!

Jorricks commented Aug 13, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mariocj89 commented Aug 15, 2022

Uh oh!

sfc-gh-izinkovsky commented Oct 4, 2022

Uh oh!

Jorricks commented Oct 5, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

11 participants

pganssle commented May 20, 2021 •

edited

Loading

Jorricks commented Jul 25, 2022 •

edited

Loading

Jorricks commented Aug 11, 2022 •

edited

Loading

Jorricks commented Aug 13, 2022 •

edited

Loading