Skip to content

Io.ascii test failures (too many open files) on Mac/Python 3.3 after #2969 #3018

Closed

Description

@taldcroft

When testing with Python 3.3 / Mac there I'm seeing consistent test failures ending with OSError: [Errno 24] Too many open files. This seems to have been introduced by #2969. I wasn't able to reproduce in limited testing outside of the test suite (i.e trying to read the same files in the same way). This doesn't happen in linux (travis) testing.

@AmrAS1
@astrofrog - have you been seeing this? Don't you normally develop/test on mac / python 3? I tried doing everything possible to make sure I'm building in a clean way.

========================================== test session starts ==========================================
platform darwin -- Python 3.3.5 -- pytest-2.5.1

Running tests with Astropy version 1.0.dev10059.
Running tests in ./astropy/io/ascii /Users/aldcroft/git/astropy/docs/io/ascii.

Platform: Darwin-13.4.0-x86_64-i386-64bit

Executable: /Users/aldcroft/anaconda/envs/py33-new/bin/python

Full Python Version: 
3.3.5 |Continuum Analytics, Inc.| (default, Sep  2 2014, 13:57:31) 
[GCC 4.2.1 (Apple Inc. build 5577)]

encodings: sys: utf-8, locale: UTF-8, filesystem: utf-8
byteorder: little
float info: dig: 15, mant_dig: 15

Numpy: 1.9.0
Scipy: not available
Matplotlib: not available
h5py: not available
Running doctests in .rst files is not supported on Python 3.x

collected 266 items 

astropy/io/ascii/daophot.py .
astropy/io/ascii/ipac.py .
astropy/io/ascii/tests/test_c_reader.py ...........................x................x.......x.x...........
astropy/io/ascii/tests/test_cds_header_from_readme.py ...
astropy/io/ascii/tests/test_compressed.py .....
astropy/io/ascii/tests/test_connect.py .............ss........
astropy/io/ascii/tests/test_fixedwidth.py ............................
astropy/io/ascii/tests/test_html.py s.ssssss.ssss..s
astropy/io/ascii/tests/test_ipac_definitions.py ...................
astropy/io/ascii/tests/test_read.py .............FF....F.F.....................................................................
astropy/io/ascii/tests/test_types.py ...
astropy/io/ascii/tests/test_write.py ..........
_________________________________ test_read_all_files_via_table[force] __________________________________

fast_reader = 'force'

    @pytest.mark.parametrize('fast_reader', [True, False, 'force'])
    def test_read_all_files_via_table(fast_reader):
        for testfile in get_testfiles():
            if testfile.get('skip'):
                print('\n\n******** SKIPPING %s' % testfile['name'])
                continue
            print('\n\n******** READING %s' % testfile['name'])
            for guess in (True, False):
                test_opts = testfile['opts'].copy()
                if 'guess' not in test_opts:
                    test_opts['guess'] = guess
                if 'Reader' in test_opts:
                    format = 'ascii.{0}'.format(test_opts['Reader']._format_name)
                    del test_opts['Reader']
                else:
                    format = 'ascii'
                if 'fast_{0}'.format(format) in core.FAST_CLASSES:
                    test_opts['fast_reader'] = fast_reader
>               table = Table.read(testfile['name'], format=format, **test_opts)

test_read.py:143: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

cls = <class 'astropy.table.table.Table'>, args = ('t/short.rdb',)
kwargs = {'format': 'ascii.rdb', 'guess': True}

    @classmethod
    def read(cls, *args, **kwargs):
        """
            Read and parse a data table and return as a Table.

            This function provides the Table interface to the astropy unified I/O
            layer.  This allows easily reading a file in many supported data formats
            using syntax such as::

              >>> from astropy.table import Table
              >>> dat = Table.read('table.dat', format='ascii')
              >>> events = Table.read('events.fits', format='fits')

            The arguments and keywords (other than ``format``) provided to this function are
            passed through to the underlying data reader (e.g. `~astropy.io.ascii.read`).
            """
>       return io_registry.read(cls, *args, **kwargs)

./astropy/table/table.py:1712: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

cls = <class 'astropy.table.table.Table'>, args = ('t/short.rdb',), kwargs = {'guess': True}
format = 'ascii.rdb', ctx = None
reader = functools.partial(<function io_read at 0x103b29c20>, 'ascii.rdb')

>   ???

./astropy/io/registry.py:330: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

format = 'rdb', filename = 't/short.rdb', kwargs = {'guess': True}, read = <function read at 0x103ae5cb0>

    def io_read(format, filename, **kwargs):
        from .ui import read
        format = re.sub(r'^ascii\.', '', format)
>       return read(filename, format=format, **kwargs)

./astropy/io/ascii/connect.py:36: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

table = 't/short.rdb', guess = True, kwargs = {'fill_values': [('', '0')], 'format': 'rdb'}
new_kwargs = {'Outputter': <class 'astropy.io.ascii.core.TableOutputter'>, 'Reader': <class 'astropy.io.ascii.basic.Rdb'>, 'fill_values': [('', '0')]}
fast_reader_param = True, format = 'rdb', Reader = <class 'astropy.io.ascii.basic.Rdb'>

    def read(table, guess=None, **kwargs):
        """Read the input ``table`` and return the table.  Most of
        the default behavior for various parameters is determined by the Reader
        class.

        :param table: input table (file name, file-like object, list of strings, or single newline-separated string)
        :param guess: try to guess the table format (default=True)
        :param format: input table format
        :param Inputter: Inputter class
        :param Outputter: Outputter class (default=TableOutputter)
        :param delimiter: column delimiter string
        :param comment: regular expression defining a comment line in table
        :param quotechar: one-character string to quote fields containing special characters
        :param header_start: line index for the header line not counting comment lines
        :param data_start: line index for the start of data not counting comment lines
        :param data_end: line index for the end of data (can be negative to count from end)
        :param converters: dict of converters
        :param data_Splitter: Splitter class to split data columns
        :param header_Splitter: Splitter class to split header columns
        :param names: list of names corresponding to each data column
        :param include_names: list of names to include in output (default=None selects all names)
        :param exclude_names: list of names to exlude from output (applied after ``include_names``)
        :param fill_values: specification of fill values for bad or missing table values (default=('', '0'))
        :param fill_include_names: list of names to include in fill_values (default=None selects all names)
        :param fill_exclude_names: list of names to exlude from fill_values (applied after ``fill_include_names``)
        :param fast_reader: whether to use the C engine, can also be a dict with options which default to False (default=True)
        :param Reader: Reader class (DEPRECATED) (default=``ascii.Basic``)
        """

        if 'fill_values' not in kwargs:
            kwargs['fill_values'] = [('', '0')]

        # If an Outputter is supplied in kwargs that will take precedence.
        new_kwargs = {}
        new_kwargs['Outputter'] = core.TableOutputter
        fast_reader_param = kwargs.get('fast_reader', True)
        if 'Outputter' in kwargs: # user specified Outputter, not supported for fast reading
            fast_reader_param = False
        format = kwargs.get('format')
        new_kwargs.update(kwargs)

        # Get the Reader class based on possible format and Reader kwarg inputs.
        Reader = _get_format_class(format, kwargs.get('Reader'), 'Reader')
        if Reader is not None:
            new_kwargs['Reader'] = Reader
            format = Reader._format_name

        # Remove format keyword if there, this is only allowed in read() not get_reader()
        if 'format' in new_kwargs:
            del new_kwargs['format']

        if guess is None:
            guess = _GUESS
        if guess:
>           dat = _guess(table, new_kwargs, format, fast_reader_param)

./astropy/io/ascii/ui.py:141: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

table = 't/short.rdb'
read_kwargs = {'Outputter': <class 'astropy.io.ascii.core.TableOutputter'>, 'Reader': <class 'astropy.io.ascii.basic.Rdb'>, 'fill_values': [('', '0')]}
format = 'rdb', fast_reader = True

    def _guess(table, read_kwargs, format, fast_reader):
        """Try to read the table using various sets of keyword args. First try the
        original args supplied in the read() call. Then try the standard guess
        keyword args. For each key/val pair specified explicitly in the read()
        call make sure that if there is a corresponding definition in the guess
        then it must have the same val.  If not then skip this guess."""

        # Keep a trace of all failed guesses kwarg
        failed_kwargs = []
        fast_kwargs = []

        first_kwargs = [read_kwargs.copy()]
        if fast_reader and format is not None and 'fast_{0}'.format(format) in \
                                                             core.FAST_CLASSES:
            # If a fast version of the reader is available, try that before the slow version
            fast_kwargs = read_kwargs.copy()
            fast_kwargs['Reader'] = core.FAST_CLASSES['fast_{0}'.format(format)]
            first_kwargs = [fast_kwargs] + first_kwargs

        # First try guessing
        for guess_kwargs in first_kwargs + _get_guess_kwargs_list():
            guess_kwargs_ok = True  # guess_kwargs are consistent with user_kwargs?
            for key, val in read_kwargs.items():
                # Do guess_kwargs.update(read_kwargs) except that if guess_args has
                # a conflicting key/val pair then skip this guess entirely.
                if key not in guess_kwargs:
                    guess_kwargs[key] = val
                elif val != guess_kwargs[key] and guess_kwargs != fast_kwargs:
                    guess_kwargs_ok = False
                    break

            if not guess_kwargs_ok:
                # User-supplied kwarg is inconsistent with the guess-supplied kwarg, e.g.
                # user supplies delimiter="|" but the guess wants to try delimiter=" ",
                # so skip the guess entirely.
                continue

            try:
                # If guessing will try all Readers then use strict req'ts on column names
                if 'Reader' not in read_kwargs:
                    guess_kwargs['strict_names'] = True

                reader = get_reader(**guess_kwargs)
>               dat = reader.read(table)

./astropy/io/ascii/ui.py:206: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <astropy.io.ascii.fastbasic.FastRdb object at 0x104105490>, table = 't/short.rdb'

    def read(self, table):
        """
            Read input data (file-like object, filename, list of strings, or
            single string) into a Table and return the result.
            """
        if self.comment is not None and len(self.comment) != 1:
            raise core.ParameterError("The C reader does not support a comment regex")
        elif self.data_start is None:
            raise core.ParameterError("The C reader does not allow data_start to be None")
        elif self.header_start is not None and self.header_start < 0 and \
             not isinstance(self, FastCommentedHeader):
            raise core.ParameterError("The C reader does not allow header_start to be "
                                      "negative except for commented-header files")
        elif self.data_start < 0:
            raise core.ParameterError("The C reader does not allow data_start to be negative")
        elif len(self.delimiter) != 1:
            raise core.ParameterError("The C reader only supports 1-char delimiters")
        elif len(self.quotechar) != 1:
            raise core.ParameterError("The C reader only supports a length-1 quote character")
        elif 'converters' in self.kwargs:
            raise core.ParameterError("The C reader does not support passing "
                                      "specialized converters")
        elif 'Outputter' in self.kwargs:
            raise core.ParameterError("The C reader does not use the Outputter parameter")
        elif 'Inputter' in self.kwargs:
            raise core.ParameterError("The C reader does not use the Inputter parameter")
        elif 'data_Splitter' in self.kwargs or 'header_Splitter' in self.kwargs:
            raise core.ParameterError("The C reader does not use a Splitter class")

        self.engine = cparser.CParser(table, self.strip_whitespace_lines,
                                      self.strip_whitespace_fields,
                                      delimiter=self.delimiter,
                                      header_start=self.header_start,
                                      comment=self.comment,
                                      quotechar=self.quotechar,
                                      data_start=self.data_start,
                                      fill_extra_cols=self.fill_extra_cols,
>                                     **self.kwargs)

./astropy/io/ascii/fastbasic.py:86: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???

astropy/io/ascii/cparser.pyx:223: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???

astropy/io/ascii/cparser.pyx:265: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???
E   OSError: [Errno 24] Too many open files

astropy/io/ascii/cparser.pyx:124: OSError
-------------------------------------------- Captured stdout --------------------------------------------

Activity

  1. added this to the v0.4.3 milestone on Oct 6, 2014
  2. astrofrog commented on Oct 6, 2014

    @astrofrog
    Member

    I've seen that too but I thought it was just because I have too much going on at once on my computer ;)

  3. taldcroft commented on Oct 6, 2014

    @taldcroft
    MemberAuthor

    I actually rebooted my laptop to rule out the "too-much-going-on" idea.

  4. mdmueller commented on Oct 7, 2014

    @mdmueller
    Contributor

    Unfortunately I don't have regular access to a Mac at the moment, but I'll definitely test this out when I get the chance. Just took a look at the code and found that the memory map isn't being closed before closing the file--maybe this is the issue? I added a line to close the memory map in this branch.

  5. astrofrog commented on Oct 7, 2014

    @astrofrog
    Member

    @AmrAS1 - I'll test it out, thanks!

  6. astrofrog commented on Oct 7, 2014

    @astrofrog
    Member

    It still fails, but interestingly I saw the following warnings in the log:

    astropy/io/ascii/tests/test_compressed.py ..............................FF/Users/tom/Library/Python/3.4/lib/python/site-packages/py/_code/source.py:367: ResourceWarning: unclosed file <_io.FileIO name='/Users/tom/Dropbox/Code/development/Astropy/astropy/astropy/io/ascii/tests/t/short.rdb.bz2' mode='rb'>
    FFFFF/Users/tom/Library/Python/3.4/lib/python/site-packages/py/_code/source.py:367: ResourceWarning: unclosed file <_io.FileIO name='/Users/tom/Dropbox/Code/development/Astropy/astropy/astropy/io/ascii/tests/t/ipac.dat.bz2' mode='rb'>
    FFFFFFFFFFFFF
    astropy/io/ascii/tests/test_connect.py FFF/Users/tom/Library/Python/3.4/lib/python/site-packages/_pytest/config.py:687: ResourceWarning: unclosed file <_io.FileIO name='/Users/tom/Dropbox/Code/development/Astropy/astropy/astropy/io/ascii/tests/t/ipac.dat.bz2' mode='rb'>```
    
  7. astrofrog commented on Oct 7, 2014

    @astrofrog
    Member

    @AmrAS1 - it might be possible to reproduce the issue on linux by reducing the default ulimit for open files? Maybe there is a difference in that for mac vs linux? On mac the default is 256.

  8. astrofrog commented on Oct 7, 2014

    @astrofrog
    Member

    I managed to list all open file handles and the list looks like:

    Python  26229  tom  152r     REG                1,2        18 45544195 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmp6x0saa8q
    Python  26229  tom  153r     REG                1,2        18 45544195 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmp6x0saa8q
    Python  26229  tom  154r     REG                1,2        18 45544196 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpim94fmep
    Python  26229  tom  155r     REG                1,2        18 45544197 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpyyi7w7q5
    Python  26229  tom  156r     REG                1,2        18 45544197 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpyyi7w7q5
    Python  26229  tom  157r     REG                1,2        18 45544198 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpvv3a4e11
    Python  26229  tom  158r     REG                1,2        18 45544199 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmp40i3iq9j
    Python  26229  tom  159r     REG                1,2        18 45544199 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmp40i3iq9j
    Python  26229  tom  160r     REG                1,2        18 45544200 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpuiyt08xp
    Python  26229  tom  161r     REG                1,2        18 45544201 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpzx9y37_q
    Python  26229  tom  162r     REG                1,2        18 45544201 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpzx9y37_q
    Python  26229  tom  163r     REG                1,2        18 45544202 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmp9g3w5dsy
    Python  26229  tom  164r     REG                1,2        47 45544203 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpd09i2mey
    Python  26229  tom  165r     REG                1,2        47 45544203 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpd09i2mey
    Python  26229  tom  166r     REG                1,2        47 45544204 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmp4zme2a9e
    Python  26229  tom  167r     REG                1,2        47 45544205 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpv8bdtfhr
    Python  26229  tom  168r     REG                1,2        47 45544205 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpv8bdtfhr
    Python  26229  tom  169r     REG                1,2        47 45544206 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmplvkyktg2
    Python  26229  tom  170r     REG                1,2        47 45544207 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpqnihi736
    Python  26229  tom  171r     REG                1,2        47 45544207 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpqnihi736
    Python  26229  tom  172r     REG                1,2        47 45544208 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmps5tbkigy
    Python  26229  tom  173r     REG                1,2        47 45544209 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmp2139jgfj
    Python  26229  tom  174r     REG                1,2        47 45544209 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmp2139jgfj
    Python  26229  tom  175r     REG                1,2        47 45544210 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmp3eo2d506
    Python  26229  tom  176r     REG                1,2        47 45544211 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpocf38809
    Python  26229  tom  177r     REG                1,2        47 45544211 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpocf38809
    Python  26229  tom  178r     REG                1,2        47 45544212 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpdd43v_b3
    Python  26229  tom  179r     REG                1,2        47 45544213 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmppukk1ofn
    Python  26229  tom  180r     REG                1,2        47 45544213 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmppukk1ofn
    Python  26229  tom  181r     REG                1,2        47 45544214 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmplnee5k_n
    Python  26229  tom  182r     REG                1,2        47 45544215 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpncqeqqcd
    Python  26229  tom  183r     REG                1,2        47 45544215 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpncqeqqcd
    Python  26229  tom  184r     REG                1,2        47 45544216 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpmxydtl48
    Python  26229  tom  185r     REG                1,2        47 45544217 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpfthz_ku6
    Python  26229  tom  186r     REG                1,2        47 45544217 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpfthz_ku6
    Python  26229  tom  187r     REG                1,2        47 45544219 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmp2exerlxz
    Python  26229  tom  188r     REG                1,2        47 45544222 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpsdimho_n
    Python  26229  tom  189r     REG                1,2        47 45544222 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpsdimho_n
    Python  26229  tom  190r     REG                1,2        47 45544223 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpmdh29yo5
    Python  26229  tom  191r     REG                1,2        47 45544224 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpzf16n8x8
    Python  26229  tom  192r     REG                1,2        47 45544224 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpzf16n8x8
    Python  26229  tom  193r     REG                1,2        47 45544225 /private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpgb8iuw6s
    

    looks like something to do with tempfile, maybe unrelated to the Cython code?

  9. astrofrog commented on Oct 7, 2014

    @astrofrog
    Member

    I think it's related to NamedTemporaryFile, investigating

  10. astrofrog commented on Oct 7, 2014

    @astrofrog
    Member

    Ok, definitely related to this block:

        with NamedTemporaryFile() as f:
            f.write(table.encode('ascii'))
            f.flush()
            t7 = ascii.read(f.name, format=format, guess=False, **kwargs)
            if parallel:
                t8 = ascii.read(f.name, format=format, guess=False, fast_reader={
                    'parallel': True}, **kwargs)
    
  11. astrofrog commented on Oct 7, 2014

    @astrofrog
    Member

    The namedtemporaryfile was a red herring, it's still happening inside the fast reader.

  12. astrofrog commented on Oct 7, 2014

    @astrofrog
    Member

    @AmrAS1 - I managed to get errors that might be helpful!

    /Users/tom/Dropbox/Code/development/Astropy/astropy/astropy/io/ascii/tests/test_c_reader.py:76: ResourceWarning: unclosed file <_io.TextIOWrapper name='/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpvwnm0giv' mode='r' encoding='UTF-8'>
      'parallel': True}, **kwargs)
    .Exception ignored in: 'astropy.io.ascii.cparser.FileString.__dealloc__'
    BufferError: cannot close exported pointers exist
    /Users/tom/Dropbox/Code/development/Astropy/astropy/astropy/io/ascii/tests/test_c_reader.py:73: ResourceWarning: unclosed file <_io.TextIOWrapper name='/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpq7wbms4_' mode='r' encoding='UTF-8'>
      t7 = ascii.read(fname, format=format, guess=False, **kwargs)
    .Exception ignored in: 'astropy.io.ascii.cparser.FileString.__dealloc__'
    BufferError: cannot close exported pointers exist
    

    Note the BufferError: cannot close exported pointers exist

  13. astrofrog commented on Oct 14, 2014

    @astrofrog
    Member

    Fixed in #3026

  14. modified the milestones: v1.0.0, v0.4.3 on Oct 14, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions