Add FixedWidthTwoLine Reader to guess list - #3109

A lot of the tests appear to be erroring, so I think this will need another pass.

@taldcroft It now passes locally for me, let's see what Travis says.
However, if you have time it would be great if you could have a look at the implementation already - I don't want to fine-tune details that you veto anyway.

In my opinion, the most tricky test is in this file: https://github.com/astropy/astropy/blob/master/docs/io/ascii/index.rst
It has the following table:

>>> lines = ['objID                   & osrcid            & xsrcid       ',
...          '----------------------- & ----------------- & -------------',
...          '              277955213 & S000.7044P00.7513 & XS04861B6_005',
...          '              889974380 & S002.9051P14.7003 & XS03957B7_004']

The problem here is that the second line contains three characters (- &) and not just two (-&). I decided to treat extra white space just the same as -. That's not clean (you could argue that the table above is not well defined and should not be read), but I think practicality beats purity here. White space is common and the potential for confusion is low.

Alternatively, I could role back two commits and fix the second line in this example to '------------------------&-------------------&--------------',.

@taldcroft All the failures are in python 3. I've not used python 3 yet, do you see where the problem is?

@hamogu - nothing jumping out at me. You probably need to install Py3 and play with it. Fortunately this is pretty trivial with anaconda.

I think the issue is that when guessing, all actual exceptions are silenced so it makes it harder to debug.

The problem with the Py3 failures stems from cases where a file object is passed in, and is related to get_readable_fileobj. In Py3 if the first guess fails then the file object ends up being closed [EDIT: not just the output of get_readable_fileobj, but the original file object]. I haven't been able to precisely isolate the difference between 2 and 3 here, but the following problem is currently in master for Py3 but not Py2:

In [1]: from astropy.io.ascii import read
In [2]: fd = open('astropy/io/ascii/tests/t/simple.txt', 'rb')
In [3]: read(fd, fast_reader=False, guess=True)
---------------------------------------------------------------------------
InconsistentTableError                    Traceback (most recent call last)

After the first failed guess fd is closed so that all subsequent guesses fail. I think that adding the FixedTwoLineHeader to the guess list generates an initial failure, which is why the tests are now failing.

Maybe if guessing is enabled and a file object is passed, that it should just be read at a high level (in ui.py). Otherwise I think the current code logic might be re-reading the file for each format (at least in Py2 where it works properly).

That makes sense. The test failures are all cases where the table would be read by the first element in the guessing before this patch. So, in the process of implementing this we have uncovered a bug different from what I set out to do originally.
This bug should also be fixed to make guessing available for file objects in python3 (probably such a rare situation that this bug has not shown up before) and unfortunately, this new bug is a show-stopper for merging this.

I think it would be best if we found a quick workaround, not necessarily an in-depth fix.
We should overhaul the way the file is read for the guessing process anyway, to try the guessing on the first 100 lines of the table for increased performance (and only try reading the full table if that passes).
@taldcroft floated that idea in the discussion of a different issue (I cannot quite remember which one) and I think it makes total sense. However, that's clearly beyond the scope of the current PR.

After tinkering with the code for a few hours, I still don't understand why calling the context manager a second time closes the file:

>>> fd = open('/home/moritz/astropy/astropy/io/ascii/tests/t/simple.txt', 'rb')
>>> with get_readable_fileobj(fd) as file_obj:
...    table = file_obj.read()
... 
>>> with get_readable_fileobj(fd) as file_obj:
...    table = file_obj.read()
... 
ERROR: ValueError: I/O operation on closed file. [unknown]
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
ValueError: I/O operation on closed file.

This might be related to #3013?

The problem here should be fixed by #3132.

@hamogu - I merged #3132 and rebased (and fixed a trivial merge conflict) and tests are passing on Py3 now for me.

I pushed the rebased version so I can see if Travis likes it.

@taldcroft : Travis passed.