Importing astropy.coordinates fails with python -OO flag

Description

I am seeing an issue very similar to this: #10361

Issue 10361 was for astropy.units. I now see the same issue with astropy.coordinates.

Expected behavior

I expect to be able to import astropy.coordinates in a .py file and run the file in python.

How to Reproduce

Steps to Reproduce
run python -OO test_script.py, where test_script.py is below:

import astropy.coordinates as apcoord
pass

Versions

Windows-10-10.0.19045-SP0
Python 3.11.2 (tags/v3.11.2:878ead1, Feb 7 2023, 16:38:35) [MSC v.1934 64 bit (AMD64)]
astropy 5.3
Numpy 1.25.0
pyerfa 2.0.0.3
Scipy 1.11.1
Matplotlib 3.7.2

Problem reproduced on Centos7/ Python3.9

Welcome to Astropy 👋 and thank you for your first issue!

A project member will respond to you as soon as possible; in the meantime, please double-check the guidelines for submitting issues and make sure you've provided the requested details.

GitHub issues in the Astropy repository are used to track bug reports and feature requests; If your issue poses a question about how to use Astropy, please instead raise your question in the Astropy Discourse user forum and close this issue.

If you feel that this issue has not been responded to in a timely manner, please send a message directly to the development mailing list. If the issue is urgent or sensitive in nature (e.g., a security vulnerability) please send an e-mail directly to the private e-mail [email protected].

Hi. Can you please share the exact traceback? How did you install astropy?

Is it from here or you see something different?

astropy/astropy/io/ascii/core.py

Line 1233 in 2ba1afc

func.__doc__ += inspect.cleandoc(cls.__doc__).strip()

(added in #8255 from 5 years ago)

❯ python -OO -c "import astropy.coordinates"
Traceback (most recent call last):
...
  File "/home/simon/dev/astropy/astropy/astropy/table/__init__.py", line 118, in
 <module>
    import astropy.io.ascii.connect
  File "/home/simon/dev/astropy/astropy/astropy/io/ascii/__init__.py", line 8, in <module>
    from .basic import (
  File "/home/simon/dev/astropy/astropy/astropy/io/ascii/basic.py", line 44, in <module>
    class Basic(core.BaseReader):
  File "/home/simon/dev/astropy/astropy/astropy/io/ascii/core.py", line 1233, in __init__
    func.__doc__ += inspect.cleandoc(cls.__doc__).strip()
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/simon/.pyenv/versions/3.11.0/lib/python3.11/inspect.py", line 873, in cleandoc
    lines = doc.expandtabs().split('\n')
            ^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'expandtabs'

With -OO docstrings are removed so __doc__ is set to None.

This looks to be related to the dynamic generation of the io.ascii doc strings. Is there a way to know in the code that the -OO flag is in effect and not generate doc strings?

Yes there is: sys.flags.optimize.

Thanks for the quick response from everyone.

I installed astropy v 5.3 using pip.

Traceback (same as others posted):

>python -OO test_script.py
Traceback (most recent call last):
  File "c:\Testing\test_script.py", line 1, in <module>
    import astropy.coordinates as apcoord
  File "C:\Users\x\AppData\Local\Programs\Python\Python311\Lib\site-packages\astropy\coordinates\__init__.py", line 13, in <module>
    from .builtin_frames import *
  File "C:\Users\x\AppData\Local\Programs\Python\Python311\Lib\site-packages\astropy\coordinates\builtin_frames\__init__.py", line 30, in <module>
    from .cirs import CIRS
  File "C:\Users\x\AppData\Local\Programs\Python\Python311\Lib\site-packages\astropy\coordinates\builtin_frames\cirs.py", line 8, in <module>
    from .utils import DEFAULT_OBSTIME, EARTH_CENTER
  File "C:\Users\x\AppData\Local\Programs\Python\Python311\Lib\site-packages\astropy\coordinates\builtin_frames\utils.py", line 16, in <module>
    from astropy.utils import iers
  File "C:\Users\x\AppData\Local\Programs\Python\Python311\Lib\site-packages\astropy\utils\iers\__init__.py", line 1, in <module>
    from .iers import *
  File "C:\Users\x\AppData\Local\Programs\Python\Python311\Lib\site-packages\astropy\utils\iers\iers.py", line 23, in <module>
    from astropy.table import MaskedColumn, QTable
  File "C:\Users\x\AppData\Local\Programs\Python\Python311\Lib\site-packages\astropy\table\__init__.py", line 118, in <module>
    import astropy.io.ascii.connect
  File "C:\Users\x\AppData\Local\Programs\Python\Python311\Lib\site-packages\astropy\io\ascii\__init__.py", line 8, in <module>
    from .basic import (
  File "C:\Users\x\AppData\Local\Programs\Python\Python311\Lib\site-packages\astropy\io\ascii\basic.py", line 44, in <module>
    class Basic(core.BaseReader):
  File "C:\Users\x\AppData\Local\Programs\Python\Python311\Lib\site-packages\astropy\io\ascii\core.py", line 1233, in __init__
    func.__doc__ += inspect.cleandoc(cls.__doc__).strip()
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\x\AppData\Local\Programs\Python\Python311\Lib\inspect.py", line 873, in cleandoc
    lines = doc.expandtabs().split('\n')
            ^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'expandtabs'

Point of interest:
sys.flags.optimize returns 0, 1 or 2
In extern/ply/yacc.py line 3217, and utils/parsing.py line 152, optimize is true/false

Would #15037 work for you all?

Your fix passes my basic tests. I'll do a lot more testing between now and Monday to make sure it caught everything. Thanks for fixing this so quickly.

I ran a slightly more complex test using the patched Astropy code with the newly patched Nuitka code (not yet in version available form pip, see Nuitka/Nuitka#2313) and got the same error message I got last week. The original instructions I provided to reproduce the issue were too simplified to show the full scope of the problem. Or maybe this is a separate issue.

Slightly more complex test instructions:
Create minimalTest.py with the following text:

import sys
print(sys.flags)
import astropy.units
import astropy.coordinates as ascord
print('hello world')

Confirm it works in python. Should output sys.flags optimize=2 and 'hello world.'
python3 -OO minimalTest.py
With the fix https://github.com/astropy/astropy/pull/15037/files, the only change without the -OO flag is that optimize=0.

Compile using Nuitka and run the resulting executable.

python3 -m nuitka --standalone --python-flag=-OO  minimalTest.py
./minimalTest.dist/minimalTest.bin

I observed the same result with and without "--python-flag=-OO," with the exception of the optimize flag.

Output of minimalTest.bin:

sys.flags(debug=0, inspect=0, interactive=0, optimize=1, dont_write_bytecode=0, no_user_site=1, no_site=1, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0, dev_mode=False, utf8_mode=1)
ERROR: No token list is defined
Traceback (most recent call last):
  File "/Testing/minimalTest.dist/astropy/units/format/generic.py", line 587, in _do_parse
  File "/Testing/minimalTest.dist/astropy/units/format/generic.py", line 483, in _parse_unit
ValueError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Testing/minimalTest.dist/astropy/units/core.py", line 2098, in __call__
  File "/Testing/minimalTest.dist/astropy/units/format/generic.py", line 571, in parse
  File "/Testing/minimalTest.dist/astropy/units/format/generic.py", line 590, in _do_parse
  File "/Testing/minimalTest.dist/astropy/utils/decorators.py", line 747, in __get__
  File "/Testing/minimalTest.dist/astropy/units/format/generic.py", line 72, in _parser
  File "/Testing/minimalTest.dist/astropy/units/format/generic.py", line 448, in _make_parser
  File "/Testing/minimalTest.dist/astropy/utils/parsing.py", line 148, in yacc
  File "/Testing/minimalTest.dist/astropy/extern/ply/yacc.py", line 3282, in yacc
astropy.extern.ply.yacc.YaccError: Unable to build parser

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Testing/minimalTest.dist/minimalTest.py", line 3, in <module>
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "/Testing/minimalTest.dist/astropy/units/__init__.py", line 25, in <module astropy.units>
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "/Testing/minimalTest.dist/astropy/units/astrophys.py", line 66, in <module astropy.units.astrophys>
  File "/Testing/minimalTest.dist/astropy/constants/constant.py", line 42, in wrapper
  File "/Testing/minimalTest.dist/astropy/units/quantity.py", line 1011, in unit
  File "/Testing/minimalTest.dist/astropy/utils/decorators.py", line 837, in __get__
  File "/Testing/minimalTest.dist/astropy/constants/constant.py", line 208, in _unit
  File "/Testing/minimalTest.dist/astropy/units/core.py", line 2121, in __call__
ValueError: 'm / (s)' did not parse as unit: Unable to build parser If this is meant to be a custom unit, define it with 'u.def_unit'. To have it recognized inside a file reader or other code, enable it with 'u.add_enabled_units'. For details, see https://docs.astropy.org/en/latest/units/combining_and_defining.html

This error message looks very similar to #10361, but I think today's fix should have resolved the related issue.

At this point, I don't know whether the problem is in Astropy or Nuitka.
Does Astropy require optimize flag=2 when python-flag = -OO? Currently Nuitka sets optimize=1 for flag -OO. (See Nuitka/Nuitka#2313)

Does Astropy require optimize flag=2 when python-flag = -OO? Currently Nuitka sets optimize=1 for flag -OO

Maybe... I cannot reproduce this error with just python -OO -c "import astropy.units; import astropy.coordinates as ascord". My Python came with Miniconda. Does that error go away if you have Nuitka set optimize=2?

Ply is bundled and actually not developed by us. And I see that we do have the latest stable from http://www.dabeaz.com/ply/ .

The newest updates to Nuitka source allowed for testing with optimize=2. This did not clear the error message I posted above.

Kayhayen posted a few suggestions for updates to astropy code here: Nuitka/Nuitka#2313. I'm hoping someone on the Astropy team who is a better programmer than me can review his comments.

Ply has new updates here: https://github.com/dabeaz/ply
The updated version doesn't look like a drop in replacement.

If no one is working on this already, I'll try updating to the new ply code and see if that helps.

@mhvk

@mhvk , what do you think? Are we ready for a ply upgrade?

Let me see if this works before committing to an upgrade. I don't have high hopes that this will fix my problem, but I'm out of ideas of things to try that I know how to implement.

Note that we discussed upgrading PLY before, but apparently this is not trivial because they changed their implementation quite drastically: #13399. Given more recent changes in PLY, it may be less work than before to adapt our usage to it (but that's better for discussion in #13399 itself).

Navigation Menu

Repository navigation

Importing astropy.coordinates fails with python -OO flag #15028

Description

Expected behavior

How to Reproduce

Versions

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Importing astropy.coordinates fails with python -OO flag #15028

Description

Description

Expected behavior

How to Reproduce

Versions

Activity

github-actions commented on Jul 5, 2023

pllim commented on Jul 6, 2023

pllim commented on Jul 6, 2023

saimn commented on Jul 6, 2023

taldcroft commented on Jul 6, 2023

eerovaher commented on Jul 6, 2023

MatCat776 commented on Jul 6, 2023

MatCat776 commented on Jul 6, 2023

pllim commented on Jul 6, 2023

MatCat776 commented on Jul 6, 2023

MatCat776 commented on Jul 6, 2023

pllim commented on Jul 7, 2023

MatCat776 commented on Jul 7, 2023

MatCat776 commented on Jul 10, 2023

pllim commented on Jul 10, 2023

MatCat776 commented on Jul 10, 2023

mhvk commented on Jul 10, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions