[Python][CI] Failing test_dateutil_tzinfo_to_string due to new release of python-dateutil

Describe the bug, including details regarding any error messages, version, and platform.

test_dateutil_tzinfo_to_string started failing with:

         tz = dateutil.tz.gettz('Europe/Paris')
>       assert pa.lib.tzinfo_to_string(tz) == 'Europe/Paris'
E       AssertionError: assert 'Europe/Monaco' == 'Europe/Paris'
E         - Europe/Paris
E         + Europe/Monaco

see: https://github.com/ursacomputing/crossbow/actions/runs/8103088008/job/22164107523#step:6:4305.

This is most probably due to new release of python-dateutil package.

The test is failing on the CI now also, with dateutil 2.9.0: https://ci.appveyor.com/project/ApacheSoftwareFoundation/arrow/builds/49322144?fullLog=true

Upstream issue: dateutil/dateutil#1353

Component(s)

Continuous Integration, Python

@raulcd

cc @raulcd

I can't reproduce this on Linux, so most probably a Windows issue (or related to the exact tzdata being used there, since for me locally it's using the system provided data)

Might be good to report upstream?

The logic we use to convert a dateutil tz to a string:

arrow/python/pyarrow/src/arrow/python/datetime.cc

Lines 554 to 568 in 2b194ad

    
           // if dateutil is installed and tzinfo is an instance of dateutil.tz.tzfile 
        
           if (module_dateutil.obj() != nullptr && 
        
               PyObject_IsInstance(tzinfo, class_tzfile.obj())) { 
        
             OwnedRef _filename(PyObject_GetAttrString(tzinfo, "_filename")); 
        
             RETURN_IF_PYERROR(); 
        
             std::string result; 
        
             RETURN_NOT_OK(internal::PyUnicode_AsStdString(_filename.obj(), &result)); 
        
             // _filename returns a full path in general ('/usr/share/zoneinfo/Europe/Paris') 
        
             // or POSIX name on Windows ('Europe/Paris') - we need a substring in first case 
        
             std::size_t pos = result.find("zoneinfo/"); 
        
             if (pos != std::string::npos) { 
        
               return result.substr(pos + 9); 
        
             } 
        
             return result; 
        
           }

(so it might also be that something changed under the hood in dateutil that messes that u. It would be good to verify if the actual tz object in the snippet above, before conversion to a string by pyarrow, also shows "Europe/Monaco")

Agree with you Joris. I would first like to check the behaviour of dateutil, as you mention, but also can't do that on my M1. I could in any case open an issue upstream before, if we think that makes sense.

The same issue is being reproduced on my PR on the conda feedstock on Windows:
https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=888527&view=logs&j=ab98fee7-5bc6-5c2f-a410-3ab9b2f2e8ca&t=472408ae-fc68-5791-981c-69ea41d2d692&l=38399

    def test_dateutil_tzinfo_to_string():
        pytest.importorskip("dateutil")
        import dateutil.tz
    
        tz = dateutil.tz.UTC
        assert pa.lib.tzinfo_to_string(tz) == 'UTC'
        tz = dateutil.tz.gettz('Europe/Paris')
>       assert pa.lib.tzinfo_to_string(tz) == 'Europe/Paris'
E       AssertionError: assert 'Europe/Monaco' == 'Europe/Paris'
E         - Europe/Paris
E         + Europe/Monaco

Edit: added on Windows

A temporary skip was added in #40486

This was reported upstream: dateutil/dateutil#1353

Navigation Menu

Repository navigation

[Python][CI] Failing test_dateutil_tzinfo_to_string due to new release of python-dateutil #40337

Describe the bug, including details regarding any error messages, version, and platform.

Component(s)

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	// if dateutil is installed and tzinfo is an instance of dateutil.tz.tzfile
	if (module_dateutil.obj() != nullptr &&
	PyObject_IsInstance(tzinfo, class_tzfile.obj())) {
	OwnedRef _filename(PyObject_GetAttrString(tzinfo, "_filename"));
	RETURN_IF_PYERROR();
	std::string result;
	RETURN_NOT_OK(internal::PyUnicode_AsStdString(_filename.obj(), &result));
	// _filename returns a full path in general ('/usr/share/zoneinfo/Europe/Paris')
	// or POSIX name on Windows ('Europe/Paris') - we need a substring in first case
	std::size_t pos = result.find("zoneinfo/");
	if (pos != std::string::npos) {
	return result.substr(pos + 9);
	}
	return result;
	}

[Python][CI] Failing test_dateutil_tzinfo_to_string due to new release of python-dateutil #40337

Description

Describe the bug, including details regarding any error messages, version, and platform.

Component(s)

Activity

AlenkaF commented on Mar 4, 2024

jorisvandenbossche commented on Mar 4, 2024

jorisvandenbossche commented on Mar 4, 2024

AlenkaF commented on Mar 4, 2024

raulcd commented on Mar 4, 2024

jorisvandenbossche commented on Mar 12, 2024

rok commented on Mar 19, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions