Skip to content

When compiled, astropy.extern.ply.yacc.YaccError: Unable to build parser #15069

Open

Description

@MatCat776

Description

The problem was initial reported as part of #15028, but has a different root cause, so I'm making a separate ticket.

Expected behavior

I should be able to write code that uses Astropy, and compile my code using Nuitka.

How to Reproduce

Create minimalTest.py with the following text:

import astropy.units
import astropy.coordinates as ascord
print('hello world')

Confirm you got the fix for #15028. Should output 'hello world.'
python3 -OO minimalTest.py
This is only necessary until the fix is in the released version.

Install Nuitka.

git clone https://github.com/Nuitka/Nuitka
python3 setup.py install

Lines above can be replaced with pip3 install nuitka once the patch from Nuitka/Nuitka#2313 is in their release branch.
Compile using Nuitka and run the resulting executable.

pip3 install patchelf
python3 -m nuitka --standalone minimalTest.py
./minimalTest.dist/minimalTest.bin

Output of minimalTest.bin:

ERROR: No token list is defined
Traceback (most recent call last):
  File "/Testing/minimalTest.dist/astropy/units/format/generic.py", line 587, in _do_parse
  File "/Testing/minimalTest.dist/astropy/units/format/generic.py", line 483, in _parse_unit
ValueError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Testing/minimalTest.dist/astropy/units/core.py", line 2098, in __call__
  File "/Testing/minimalTest.dist/astropy/units/format/generic.py", line 571, in parse
  File "/Testing/minimalTest.dist/astropy/units/format/generic.py", line 590, in _do_parse
  File "/Testing/minimalTest.dist/astropy/utils/decorators.py", line 747, in __get__
  File "/Testing/minimalTest.dist/astropy/units/format/generic.py", line 72, in _parser
  File "/Testing/minimalTest.dist/astropy/units/format/generic.py", line 448, in _make_parser
  File "/Testing/minimalTest.dist/astropy/utils/parsing.py", line 148, in yacc
  File "/Testing/minimalTest.dist/astropy/extern/ply/yacc.py", line 3282, in yacc
astropy.extern.ply.yacc.YaccError: Unable to build parser

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Testing/minimalTest.dist/minimalTest.py", line 3, in <module>
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "/Testing/minimalTest.dist/astropy/units/__init__.py", line 25, in <module astropy.units>
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "/Testing/minimalTest.dist/astropy/units/astrophys.py", line 66, in <module astropy.units.astrophys>
  File "/Testing/minimalTest.dist/astropy/constants/constant.py", line 42, in wrapper
  File "/Testing/minimalTest.dist/astropy/units/quantity.py", line 1011, in unit
  File "/Testing/minimalTest.dist/astropy/utils/decorators.py", line 837, in __get__
  File "/Testing/minimalTest.dist/astropy/constants/constant.py", line 208, in _unit
  File "/Testing/minimalTest.dist/astropy/units/core.py", line 2121, in __call__
ValueError: 'm / (s)' did not parse as unit: Unable to build parser If this is meant to be a custom unit, define it with 'u.def_unit'. To have it recognized inside a file reader or other code, enable it with 'u.add_enabled_units'. For details, see https://docs.astropy.org/en/latest/units/combining_and_defining.html

This error message looks very similar to #10361, but I think the fix for 15037 should have resolved the related issue.

For those following the previous thread, Nuitka fixed their optimize flag problem, and that did not resolve my issue.

Kayhayen has some suggestions in this thread that I don't know exactly how to implement, but I'm not the best Python programmer: Nuitka/Nuitka#2313

Versions

Linux-5.18.7-1.el7.elrepo.x86_64-x86_64-with-glibc2.17
Python 3.9.10 (main, Jun 29 2023, 21:55:15)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-44)]
astropy 6.0.dev396+g7e9f51e (need this to get -OO fix, but get similar errors with stable release)
Numpy 1.25.0
pyerfa 2.0.0.3
Traceback (most recent call last):
File "/generator/Testing/minimalTest.py", line 9, in
import scipy; print("Scipy", scipy.version)
ModuleNotFoundError: No module named 'scipy'

Activity

  1. pllim commented on Jul 13, 2023

    @pllim
    Member

    Well, at least we got past the coordinates import error for __doc__. 😬

  2. pllim commented on Jul 13, 2023

    @pllim
    Member

    The person mentioned a piece of code in Nuitka/Nuitka#2313 that I believe was added in #11227 by @bmerry for thread-safety; so perhaps bmerry can shed some light on this? 🙏

  3. bmerry commented on Jul 13, 2023

    @bmerry
    Contributor

    I'm not familiar with Nuitka at all, but from glancing at the discussion, the code that's particularly problematic (get_caller_module_dict) is in the vendored ply module (and there are two copies of it, one in lex.py and one in yacc.py). My changes in #11227 does some patching of those functions, but it wasn't clear to me whether that patching is what's causing problems for Nuitka, or the way ply does introspection (the patching was done to avoid introducing changes in the vendored copy of the ply package).

  4. MatCat776 commented on Jul 14, 2023

    @MatCat776
    ContributorAuthor

    No solution yet, but some interesting facts that may help with debugging.

    I printed out the value of pdict after astropy/astropy/extern/ply/yacc.py line 3245. The value of pdict is slightly different depending on whether I'm running normal python or a bin compiled by Nuitka. Note that in the normal python method, tokens is defined towards the bottom, but this section is missing from the compiled bin version.

    Normal python use:
    debug out: pdict

    {'__name__': 'astropy.units.format.generic', '__doc__': '\nHandles a "generic" string format for units\n', '__package__': 'astropy.units.format', '__loader__': <_frozen_importlib_external.SourceFileLoader object at 0x7f4fe817cc70>, '__spec__': ModuleSpec(name='astropy.units.format.generic', loader=<_frozen_importlib_external.SourceFileLoader object at 0x7f4fe817cc70>, origin='/generator/tmp/astropy/astropy/units/format/generic.py'), '__file__': '/generator/tmp/astropy/astropy/units/format/generic.py', '__cached__': '/generator/tmp/astropy/astropy/units/format/__pycache__/generic.cpython-39.pyc', '__builtins__': {'__name__': 'builtins', '__doc__': "Built-in functions, exceptions, and other objects.\n\nNoteworthy: None is the `nil' object; Ellipsis represents `...' in slices.", '__package__': '', '__loader__': <class '_frozen_importlib.BuiltinImporter'>, '__spec__': ModuleSpec(name='builtins', loader=<class '_frozen_importlib.BuiltinImporter'>, origin='built-in'), '__build_class__': <built-in function __build_class__>, '__import__': <built-in function __import__>, 'abs': <built-in function abs>, 'all': <built-in function all>, 'any': <built-in function any>, 'ascii': <built-in function ascii>, 'bin': <built-in function bin>, 'breakpoint': <built-in function breakpoint>, 'callable': <built-in function callable>, 'chr': <built-in function chr>, 'compile': <built-in function compile>, 'delattr': <built-in function delattr>, 'dir': <built-in function dir>, 'divmod': <built-in function divmod>, 'eval': <built-in function eval>, 'exec': <built-in function exec>, 'format': <built-in function format>, 'getattr': <built-in function getattr>, 'globals': <built-in function globals>, 'hasattr': <built-in function hasattr>, 'hash': <built-in function hash>, 'hex': <built-in function hex>, 'id': <built-in function id>, 'input': <built-in function input>, 'isinstance': <built-in function isinstance>, 'issubclass': <built-in function issubclass>, 'iter': <built-in function iter>, 'len': <built-in function len>, 'locals': <built-in function locals>, 'max': <built-in function max>, 'min': <built-in function min>, 'next': <built-in function next>, 'oct': <built-in function oct>, 'ord': <built-in function ord>, 'pow': <built-in function pow>, 'print': <built-in function print>, 'repr': <built-in function repr>, 'round': <built-in function round>, 'setattr': <built-in function setattr>, 'sorted': <built-in function sorted>, 'sum': <built-in function sum>, 'vars': <built-in function vars>, 'None': None, 'Ellipsis': Ellipsis, 'NotImplemented': NotImplemented, 'False': False, 'True': True, 'bool': <class 'bool'>, 'memoryview': <class 'memoryview'>, 'bytearray': <class 'bytearray'>, 'bytes': <class 'bytes'>, 'classmethod': <class 'classmethod'>, 'complex': <class 'complex'>, 'dict': <class 'dict'>, 'enumerate': <class 'enumerate'>, 'filter': <class 'filter'>, 'float': <class 'float'>, 'frozenset': <class 'frozenset'>, 'property': <class 'property'>, 'int': <class 'int'>, 'list': <class 'list'>, 'map': <class 'map'>, 'object': <class 'object'>, 'range': <class 'range'>, 'reversed': <class 'reversed'>, 'set': <class 'set'>, 'slice': <class 'slice'>, 'staticmethod': <class 'staticmethod'>, 'str': <class 'str'>, 'super': <class 'super'>, 'tuple': <class 'tuple'>, 'type': <class 'type'>, 'zip': <class 'zip'>, '__debug__': True, 'BaseException': <class 'BaseException'>, 'Exception': <class 'Exception'>, 'TypeError': <class 'TypeError'>, 'StopAsyncIteration': <class 'StopAsyncIteration'>, 'StopIteration': <class 'StopIteration'>, 'GeneratorExit': <class 'GeneratorExit'>, 'SystemExit': <class 'SystemExit'>, 'KeyboardInterrupt': <class 'KeyboardInterrupt'>, 'ImportError': <class 'ImportError'>, 'ModuleNotFoundError': <class 'ModuleNotFoundError'>, 'OSError': <class 'OSError'>, 'EnvironmentError': <class 'OSError'>, 'IOError': <class 'OSError'>, 'EOFError': <class 'EOFError'>, 'RuntimeError': <class 'RuntimeError'>, 'RecursionError': <class 'RecursionError'>, 'NotImplementedError': <class 'NotImplementedError'>, 'NameError': <class 'NameError'>, 'UnboundLocalError': <class 'UnboundLocalError'>, 'AttributeError': <class 'AttributeError'>, 'SyntaxError': <class 'SyntaxError'>, 'IndentationError': <class 'IndentationError'>, 'TabError': <class 'TabError'>, 'LookupError': <class 'LookupError'>, 'IndexError': <class 'IndexError'>, 'KeyError': <class 'KeyError'>, 'ValueError': <class 'ValueError'>, 'UnicodeError': <class 'UnicodeError'>, 'UnicodeEncodeError': <class 'UnicodeEncodeError'>, 'UnicodeDecodeError': <class 'UnicodeDecodeError'>, 'UnicodeTranslateError': <class 'UnicodeTranslateError'>, 'AssertionError': <class 'AssertionError'>, 'ArithmeticError': <class 'ArithmeticError'>, 'FloatingPointError': <class 'FloatingPointError'>, 'OverflowError': <class 'OverflowError'>, 'ZeroDivisionError': <class 'ZeroDivisionError'>, 'SystemError': <class 'SystemError'>, 'ReferenceError': <class 'ReferenceError'>, 'MemoryError': <class 'MemoryError'>, 'BufferError': <class 'BufferError'>, 'Warning': <class 'Warning'>, 'UserWarning': <class 'UserWarning'>, 'DeprecationWarning': <class 'DeprecationWarning'>, 'PendingDeprecationWarning': <class 'PendingDeprecationWarning'>, 'SyntaxWarning': <class 'SyntaxWarning'>, 'RuntimeWarning': <class 'RuntimeWarning'>, 'FutureWarning': <class 'FutureWarning'>, 'ImportWarning': <class 'ImportWarning'>, 'UnicodeWarning': <class 'UnicodeWarning'>, 'BytesWarning': <class 'BytesWarning'>, 'ResourceWarning': <class 'ResourceWarning'>, 'ConnectionError': <class 'ConnectionError'>, 'BlockingIOError': <class 'BlockingIOError'>, 'BrokenPipeError': <class 'BrokenPipeError'>, 'ChildProcessError': <class 'ChildProcessError'>, 'ConnectionAbortedError': <class 'ConnectionAbortedError'>, 'ConnectionRefusedError': <class 'ConnectionRefusedError'>, 'ConnectionResetError': <class 'ConnectionResetError'>, 'FileExistsError': <class 'FileExistsError'>, 'FileNotFoundError': <class 'FileNotFoundError'>, 'IsADirectoryError': <class 'IsADirectoryError'>, 'NotADirectoryError': <class 'NotADirectoryError'>, 'InterruptedError': <class 'InterruptedError'>, 'PermissionError': <class 'PermissionError'>, 'ProcessLookupError': <class 'ProcessLookupError'>, 'TimeoutError': <class 'TimeoutError'>, 'open': <built-in function open>, 'quit': Use quit() or Ctrl-D (i.e. EOF) to exit, 'exit': Use exit() or Ctrl-D (i.e. EOF) to exit, 'copyright': Copyright (c) 2001-2022 Python Software Foundation.
    All Rights Reserved.
    
    Copyright (c) 2000 BeOpen.com.
    All Rights Reserved.
    
    Copyright (c) 1995-2001 Corporation for National Research Initiatives.
    All Rights Reserved.
    
    Copyright (c) 1991-1995 Stichting Mathematisch Centrum, Amsterdam.
    All Rights Reserved., 'credits':     Thanks to CWI, CNRI, BeOpen.com, Zope Corporation and a cast of thousands
        for supporting Python development.  See www.python.org for more information., 'license': Type license() to see the full license text, 'help': Type help() for interactive help, or help(object) for help about object.}, 're': <module 're' from '/usr/local/lib/python3.9/re.py'>, 'unicodedata': <module 'unicodedata' from '/usr/local/lib/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so'>, 'warnings': <module 'warnings' from '/usr/local/lib/python3.9/warnings.py'>, 'Fraction': <class 'fractions.Fraction'>, 'classproperty': <class 'astropy.utils.decorators.classproperty'>, 'deprecated': <function deprecated at 0x7f5001126ee0>, 'parsing': <module 'astropy.utils.parsing' from '/generator/tmp/astropy/astropy/utils/parsing.py'>, 'did_you_mean': <function did_you_mean at 0x7f4fe8537040>, 'core': <module 'astropy.units.core' from '/generator/tmp/astropy/astropy/units/core.py'>, 'Base': <class 'astropy.units.format.base.Base'>, 'Generic': <class 'astropy.units.format.generic.Generic'>, 'Unscaled': <class 'astropy.units.format.generic.Unscaled'>, 'tokens': ('COMMA', 'DOUBLE_STAR', 'STAR', 'PERIOD', 'SOLIDUS', 'CARET', 'OPEN_PAREN', 'CLOSE_PAREN', 'FUNCNAME', 'UNIT', 'SIGN', 'UINT', 'UFLOAT'), 'p_main': <function Generic._make_parser.<locals>.p_main at 0x7f4fe80b9670>, 'p_structured_subunit': <function Generic._make_parser.<locals>.p_structured_subunit at 0x7f4fe80b9700>, 'p_structured_unit': <function Generic._make_parser.<locals>.p_structured_unit at 0x7f4fe805a670>, 'p_subunit': <function Generic._make_parser.<locals>.p_subunit at 0x7f4fe805a700>, 'p_unit': <function Generic._make_parser.<locals>.p_unit at 0x7f4fe805a790>, 'p_division_product_of_units': <function Generic._make_parser.<locals>.p_division_product_of_units at 0x7f4fe805a820>, 'p_inverse_unit': <function Generic._make_parser.<locals>.p_inverse_unit at 0x7f4fe805a8b0>, 'p_factor': <function Generic._make_parser.<locals>.p_factor at 0x7f4fe805a940>, 'p_factor_float': <function Generic._make_parser.<locals>.p_factor_float at 0x7f4fe805a9d0>, 'p_factor_int': <function Generic._make_parser.<locals>.p_factor_int at 0x7f4fe805aa60>, 'p_factor_fits': <function Generic._make_parser.<locals>.p_factor_fits at 0x7f4fe805aaf0>, 'p_product_of_units': <function Generic._make_parser.<locals>.p_product_of_units at 0x7f4fe805ab80>, 'p_unit_expression': <function Generic._make_parser.<locals>.p_unit_expression at 0x7f4fe805ac10>, 'p_unit_with_power': <function Generic._make_parser.<locals>.p_unit_with_power at 0x7f4fe805aca0>, 'p_numeric_power': <function Generic._make_parser.<locals>.p_numeric_power at 0x7f4fe805ad30>, 'p_paren_expr': <function Generic._make_parser.<locals>.p_paren_expr at 0x7f4fe805adc0>, 'p_frac': <function Generic._make_parser.<locals>.p_frac at 0x7f4fe805ae50>, 'p_sign': <function Generic._make_parser.<locals>.p_sign at 0x7f4fe805aee0>, 'p_product': <function Generic._make_parser.<locals>.p_product at 0x7f4fe805af70>, 'p_division': <function Generic._make_parser.<locals>.p_division at 0x7f4fe7a2f040>, 'p_power': <function Generic._make_parser.<locals>.p_power at 0x7f4fe7a2f0d0>, 'p_signed_int': <function Generic._make_parser.<locals>.p_signed_int at 0x7f4fe7a2f160>, 'p_signed_float': <function Generic._make_parser.<locals>.p_signed_float at 0x7f4fe7a2f1f0>, 'p_function_name': <function Generic._make_parser.<locals>.p_function_name at 0x7f4fe7a2f280>, 'p_function': <function Generic._make_parser.<locals>.p_function at 0x7f4fe7a2f310>, 'p_error': <function Generic._make_parser.<locals>.p_error at 0x7f4fe7a2f3a0>, 'cls': <class 'astropy.units.format.generic.Generic'>}
    

    Combiled bin use:
    debug out: pdict

    {'__name__': 'astropy.units.format.generic', '__doc__': '\nHandles a "generic" string format for units\n', '__package__': 'astropy.units.format', '__loader__': <class 'nuitka_module_loader'>, '__spec__': ModuleSpec(name='astropy.units.format.generic', loader=<class 'nuitka_module_loader'>, origin='/generator/tmp/Nuitka/minimalTest.dist/astropy/units/format/generic.py'), '__compiled__': __nuitka_version__(major=1, minor=8, micro=0, releaselevel='candidate', standalone=True, onefile=False, no_asserts=False, no_docstrings=False, no_annotations=False), '__builtins__': {'__name__': 'builtins', '__doc__': "Built-in functions, exceptions, and other objects.\n\nNoteworthy: None is the `nil' object; Ellipsis represents `...' in slices.", '__package__': '', '__loader__': <class '_frozen_importlib.BuiltinImporter'>, '__spec__': ModuleSpec(name='builtins', loader=<class '_frozen_importlib.BuiltinImporter'>, origin='built-in'), '__build_class__': <built-in function __build_class__>, '__import__': <built-in function __import__>, 'abs': <built-in function abs>, 'all': <built-in function all>, 'any': <built-in function any>, 'ascii': <built-in function ascii>, 'bin': <built-in function bin>, 'breakpoint': <built-in function breakpoint>, 'callable': <built-in function callable>, 'chr': <built-in function chr>, 'compile': <built-in function compile>, 'delattr': <built-in function delattr>, 'dir': <built-in function dir>, 'divmod': <built-in function divmod>, 'eval': <built-in function eval>, 'exec': <built-in function exec>, 'format': <built-in function format>, 'getattr': <built-in function getattr>, 'globals': <built-in function globals>, 'hasattr': <built-in function hasattr>, 'hash': <built-in function hash>, 'hex': <built-in function hex>, 'id': <built-in function id>, 'input': <built-in function input>, 'isinstance': <built-in function isinstance>, 'issubclass': <built-in function issubclass>, 'iter': <built-in function iter>, 'len': <built-in function len>, 'locals': <built-in function locals>, 'max': <built-in function max>, 'min': <built-in function min>, 'next': <built-in function next>, 'oct': <built-in function oct>, 'ord': <built-in function ord>, 'pow': <built-in function pow>, 'print': <built-in function print>, 'repr': <built-in function repr>, 'round': <built-in function round>, 'setattr': <built-in function setattr>, 'sorted': <built-in function sorted>, 'sum': <built-in function sum>, 'vars': <built-in function vars>, 'None': None, 'Ellipsis': Ellipsis, 'NotImplemented': NotImplemented, 'False': False, 'True': True, 'bool': <class 'bool'>, 'memoryview': <class 'memoryview'>, 'bytearray': <class 'bytearray'>, 'bytes': <class 'bytes'>, 'classmethod': <class 'classmethod'>, 'complex': <class 'complex'>, 'dict': <class 'dict'>, 'enumerate': <class 'enumerate'>, 'filter': <class 'filter'>, 'float': <class 'float'>, 'frozenset': <class 'frozenset'>, 'property': <class 'property'>, 'int': <class 'int'>, 'list': <class 'list'>, 'map': <class 'map'>, 'object': <class 'object'>, 'range': <class 'range'>, 'reversed': <class 'reversed'>, 'set': <class 'set'>, 'slice': <class 'slice'>, 'staticmethod': <class 'staticmethod'>, 'str': <class 'str'>, 'super': <class 'super'>, 'tuple': <class 'tuple'>, 'type': <class 'type'>, 'zip': <class 'zip'>, '__debug__': True, 'BaseException': <class 'BaseException'>, 'Exception': <class 'Exception'>, 'TypeError': <class 'TypeError'>, 'StopAsyncIteration': <class 'StopAsyncIteration'>, 'StopIteration': <class 'StopIteration'>, 'GeneratorExit': <class 'GeneratorExit'>, 'SystemExit': <class 'SystemExit'>, 'KeyboardInterrupt': <class 'KeyboardInterrupt'>, 'ImportError': <class 'ImportError'>, 'ModuleNotFoundError': <class 'ModuleNotFoundError'>, 'OSError': <class 'OSError'>, 'EnvironmentError': <class 'OSError'>, 'IOError': <class 'OSError'>, 'EOFError': <class 'EOFError'>, 'RuntimeError': <class 'RuntimeError'>, 'RecursionError': <class 'RecursionError'>, 'NotImplementedError': <class 'NotImplementedError'>, 'NameError': <class 'NameError'>, 'UnboundLocalError': <class 'UnboundLocalError'>, 'AttributeError': <class 'AttributeError'>, 'SyntaxError': <class 'SyntaxError'>, 'IndentationError': <class 'IndentationError'>, 'TabError': <class 'TabError'>, 'LookupError': <class 'LookupError'>, 'IndexError': <class 'IndexError'>, 'KeyError': <class 'KeyError'>, 'ValueError': <class 'ValueError'>, 'UnicodeError': <class 'UnicodeError'>, 'UnicodeEncodeError': <class 'UnicodeEncodeError'>, 'UnicodeDecodeError': <class 'UnicodeDecodeError'>, 'UnicodeTranslateError': <class 'UnicodeTranslateError'>, 'AssertionError': <class 'AssertionError'>, 'ArithmeticError': <class 'ArithmeticError'>, 'FloatingPointError': <class 'FloatingPointError'>, 'OverflowError': <class 'OverflowError'>, 'ZeroDivisionError': <class 'ZeroDivisionError'>, 'SystemError': <class 'SystemError'>, 'ReferenceError': <class 'ReferenceError'>, 'MemoryError': <class 'MemoryError'>, 'BufferError': <class 'BufferError'>, 'Warning': <class 'Warning'>, 'UserWarning': <class 'UserWarning'>, 'DeprecationWarning': <class 'DeprecationWarning'>, 'PendingDeprecationWarning': <class 'PendingDeprecationWarning'>, 'SyntaxWarning': <class 'SyntaxWarning'>, 'RuntimeWarning': <class 'RuntimeWarning'>, 'FutureWarning': <class 'FutureWarning'>, 'ImportWarning': <class 'ImportWarning'>, 'UnicodeWarning': <class 'UnicodeWarning'>, 'BytesWarning': <class 'BytesWarning'>, 'ResourceWarning': <class 'ResourceWarning'>, 'ConnectionError': <class 'ConnectionError'>, 'BlockingIOError': <class 'BlockingIOError'>, 'BrokenPipeError': <class 'BrokenPipeError'>, 'ChildProcessError': <class 'ChildProcessError'>, 'ConnectionAbortedError': <class 'ConnectionAbortedError'>, 'ConnectionRefusedError': <class 'ConnectionRefusedError'>, 'ConnectionResetError': <class 'ConnectionResetError'>, 'FileExistsError': <class 'FileExistsError'>, 'FileNotFoundError': <class 'FileNotFoundError'>, 'IsADirectoryError': <class 'IsADirectoryError'>, 'NotADirectoryError': <class 'NotADirectoryError'>, 'InterruptedError': <class 'InterruptedError'>, 'PermissionError': <class 'PermissionError'>, 'ProcessLookupError': <class 'ProcessLookupError'>, 'TimeoutError': <class 'TimeoutError'>, 'open': <built-in function open>, '__nuitka_binary_dir': '/generator/tmp/Nuitka/minimalTest.dist', '__nuitka_binary_exe': '/generator/tmp/Nuitka/minimalTest.dist/minimalTest.bin', '__nuitka_loader_type': <class 'nuitka_module_loader'>}, '__file__': '/generator/tmp/Nuitka/minimalTest.dist/astropy/units/format/generic.py', '__cached__': None, 're': <module 're' from '/generator/tmp/Nuitka/minimalTest.dist/re.py'>, 'unicodedata': <module 'unicodedata' (/generator/tmp/Nuitka/minimalTest.dist/unicodedata.so)>, 'warnings': <module 'warnings' (frozen)>, 'Fraction': <class 'fractions.Fraction'>, 'classproperty': <class 'astropy.utils.decorators.classproperty'>, 'deprecated': <compiled_function deprecated at 0x7fd38530d4f0>, 'parsing': <module 'astropy.utils.parsing' from '/generator/tmp/Nuitka/minimalTest.dist/astropy/utils/parsing.py'>, 'did_you_mean': <compiled_function did_you_mean at 0x7fd3852d3d60>, 'core': <module 'astropy.units.core' from '/generator/tmp/Nuitka/minimalTest.dist/astropy/units/core.py'>, 'Base': <class 'astropy.units.format.base.Base'>, 'Generic': <class 'astropy.units.format.generic.Generic'>, 'Unscaled': <class 'astropy.units.format.generic.Unscaled'>}
    

    Search for 'Timeout' in both error strings. (Timeout is not significant, but it gets you to the right place in the string. You can't search for a missing term.) Notice that the first significant deviation is for the value 'quit'.
    Any chance Nuitka does something odd with the value 'quit'?

  5. MatCat776 commented on Jul 14, 2023

    @MatCat776
    ContributorAuthor

    Alternate solution to consider: If Astropy is planning to update to the new version of Ply, we should probably wait for that upgrade before troubleshooting this.
    Current version of Ply is available here. https://github.com/dabeaz/ply
    Ply/Astropy discussions are spread across a few threads:
    #15028
    #13399

    I looked into updating Ply and it is a bit over my head. I'm happy to help with this transition if this is likely a good solution, but I'm not going to dive in just yet.

    Update: I looked at the source code for the current Ply and its usage of get_caller_module_dict looks quite similar to the old code.

  6. pllim commented on Jul 14, 2023

    @pllim
    Member

    #13399 has been open for a year, so I am not sure if it is high priority for us, especially when it is not trivial.

  7. MatCat776 commented on Jul 19, 2023

    @MatCat776
    ContributorAuthor

    I'm out of time to debug this. Unfortunately our solution for now is to not use the Astropy library. I do hope to come back at some point.

  8. golfinq commented on May 20, 2025

    @golfinq

    If someone could look at the work I did that would be great - as fixing this on the astropy side is easier than on the nuitka side

    Nuitka/Nuitka#2313 (comment)

  9. pllim commented on May 20, 2025

    @pllim
  10. golfinq commented on May 20, 2025

    @golfinq

    tl;dr -- What is the minimal change here that could satisfy your use case? Please advise. Thanks.

    Ply has a parameter called "module=" in yacc and lex which can be used to supply p_* and t_* definitions explicitly as opposed to relying on introspection. The most straight forward method would be to utilize this parameter in order to supply the needed definitions to the constructor. This is because compliation makes the call chain introspection unreliable.

  11. pllim commented on May 20, 2025

    @pllim
    Member

    Having glanced at the updated convo at Nuitka/Nuitka#2313 and dug up some old ply issues we have, I am not that optimistic the patch on astropy side would be simple. First of all, the one we bundle is old, but upgrading is not easy (#13399). And then as you pointed out in Nuitka/Nuitka#2313 (comment) , there are multiple places that would need refactoring. I don't even recall why it was designed this way to begin with... @mhvk , could you add more details if you remember?

    Thanks, all!

  12. mhvk commented on May 21, 2025

    @mhvk
    Contributor

    The units stuff for ply predates me. I don't think there is any sense in us adjusting our use of the old, vendored ply code; instead, we should update to more recent ply (or another solution for this type of parsing). Bottom line: I think solving this is blocked by #13399.

  13. golfinq commented on May 21, 2025

    @golfinq

    As far as replacements are concerned there is lark and pyparsing - would need to look at the token definitions more to see if either of them would work

  14. mhvk commented on May 26, 2025

    @mhvk
    Contributor

    From a look around (e.g., https://github.com/goodmami/python-parsing-benchmarks), it seems most other parsing libraries are rather slow, which makes it less clear whether they are good alternatives. Would certainly be wise to have some benchmarks first (and maybe have a look at what the other units packages use).

    On the more general issue, I think defining the PLY parser with anything resembling -OO will always fail, since the parser is defined in docstrings, and those are stripped in standard optimize mode at least.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions