Ignore errors about invalid headers when extra data are found at EOF - #11285

The reason this test works for you interactively but not in the test suite, as that fits.open() doesn't actually immediately try to read the whole file (it is lazy).

When you do this at the interactive prompt it tries to represent the resulting HDUList object; i.e. it is the calling the equivalent of

_ = fits.open('...')
repr(_)

whereas here the return value of fits.open is simply ignored and not used. Try instead calling something that will force it to read the full file, like fits.info(). See also my earlier comment on how you can rewrite this test to reproduce the problem with an arbitrary file.

Update: It turns out, ironically, calling fits.info() when the file contains trailing bytes does not reproduce the error, and instead results in the warning we want. I have to find out what is happening differently when calling fits.info().

It turns out fits.info() passes ignore_missing_end=True by default, so it doesn't reproduce the error. Instead, try something like len(fits.open(self.temp('temp.fits'))). That does the trick.

Thank you very much for all your explanations. I have modified the test accordingly.

I have modified a bit your suggestion because in my previous version (without the with block), a RessourceWarning was thrown (the file wasn't closed).

You are right, of course.

This is a perfect interpretation of what I wrote in #4351, though I think what I wrote there is not quite correct. Normally SIMPLE = is only in the first header of a FITS file. So if we were dealing with something like an accidentally truncated FITS file, the more likely thing you would see at the beginning of another header would be XTENSION =.

But I think I found a simpler solution to this problem. Instead of what you did, try moving the lines a bit below this:

        header_str = ''.join(read_blocks)
        _check_padding(header_str, actual_block_size, is_eof,
                       check_block_size=padding)

to just before this line. As far as I can tell that also solves the problem, and doesn't break any existing tests. Please try out that solution and double-check me.

Normally SIMPLE = is only in the first header of a FITS file. So if we were dealing with something like an accidentally truncated FITS file, the more likely thing you would see at the beginning of another header would be XTENSION =.

You are right. I should have thought about that! Anyway, moving the check before the if statement works like a charm.