Skip to content

Fix detection of images shorter than 80 bytes - #9553

Open
JayOfTheKeyboard wants to merge 1 commit into
Exiv2:mainfrom
JayOfTheKeyboard:fix-short-file-type-detection
Open

JayOfTheKeyboard wants to merge 1 commit into
Exiv2:mainfrom
JayOfTheKeyboard:fix-short-file-type-detection

Conversation

@JayOfTheKeyboard

Copy link
Copy Markdown

Bug

Small image files are reported as an unknown type:

$ magick -size 2x2 xc:red t.gif            # 43 bytes
$ exiv2 t.gif
Exiv2 exception in print action for file t.gif:
t.gif: The file contains data of an unknown image type

A 58-byte 1x1 BMP fails the same way. Padding the same GIF with 40 zero bytes makes it work (MIME type : image/gif).

isXmpType (src/xmpsidecar.cpp) reads 55 bytes and then up to 80. When the file is shorter it returns false without seeking back, so the stream is left at EOF. ImageFactory::getType and ImageFactory::open pass the same BasicIo to each detector in turn, and every detector registered after XMP (gif, psd, tga, bmp, jp2, the video formats, bmff) then reads nothing and returns false as well. Both early returns are hit: the GIF fails the first read, the BMP the second.

Fix

Save tell() on entry and seek back to it before both early return falses. isEpsType already does exactly this (const size_t restore = iIo.tell(); ... iIo.seek(restore, BasicIo::beg);).

The early return true for an empty sidecar (only the XMP packet header, which is what ImageFactory::create(ImageType::xmp, ...) writes) had the same problem: it left the stream at EOF even with advance == false. It now seeks back too when advance is false, matching what the end of the function does.

I put the fix in the detector rather than in the ImageFactory loops because the advance == false contract (leave the position where it was) belongs to each is*Type function, and they are also called directly through ImageFactory::checkType and from the image classes' readMetadata.

One side effect: XmpSidecar::readMetadata on a non-XMP file shorter than 80 bytes now throws kerNotAnImage ("This does not look like a XMP image") rather than kerFailedToReadImageData, because EOF is no longer set when it checks. That is the same error a longer non-XMP file already gets.

Not changed

The other fixed-length detectors (isCr2Type 16 bytes, isCrwType 14, isRw2Type 24, and so on) have the same no-rewind early return. With this fix, files of 24 bytes and up are detected; below that, isPngType's entry check throws "Failed to read input data". A valid GIF or BMP needs at least 26 bytes of headers, so I left those alone. If you would rather handle all of them at once by seeking back to the start in the ImageFactory loops, I can change it to that.

Two older things in isXmpType I noticed but left for separate PRs, since neither is needed for this fix:

  • After a short first read, strncmp compares the header against bytes of buf that were never read. MemorySanitizer reports it for a 30-byte file holding the start of the packet header.
  • For a non-XMP input that starts with a UTF-8 BOM, the final seek goes back len - start bytes, so the stream ends up 3 bytes past where it started.

Like isEpsType, the new seeks don't check the return value.

Testing

Three tests in unitTests/test_ImageFactory.cpp. The real 43-byte GIF and 58-byte BMP are byte arrays in the test, so there are no new files under test/data.

  • detectsImagesShorterThanAnXmpPacketHeaderInMemory checks getType and opens each image from memory to check the pixel size.
  • detectsImagesShorterThanAnXmpPacketHeaderInFiles does the same through FileIo, which is what exiv2 file.gif uses. It writes the bytes to std::filesystem::temp_directory_path() and removes them afterwards.
  • isXmpTypeKeepsPositionForEmptySidecar creates an empty sidecar with ImageFactory::create and checks that isXmpType(io, false) returns true with tell() still 0 and EOF not set.

All three fail without the fix. Removing one seek at a time turns a different check red each time: the empty-sidecar seek fails the third test, the first short-read seek fails the GIF checks in the first two, and the second fails the BMP checks. With the fix, all of them pass. exiv2 t.gif and exiv2 t.bmp print MIME type : image/gif / image/x-ms-bmp and the right image size.

Full ctest on Linux (GCC, Release, XMP on, curl off): all 373 unit tests and the bugfix, regression, lens, tiff and version tests pass. bashTests has two failures, io_test and stdin_test, which fail identically on main without this change on my machine (stdin_test differs in compressed iCCP bytes, from the system zlib-ng; in io_test, iotest is killed on its http://127.0.0.1 case). clang-format 20 is clean on both touched files.

Prepared with Claude Code.

JayOfTheKeyboard added a commit to JayOfTheKeyboard/JayOfTheKeyboard that referenced this pull request Oct 28, 2026
@JayOfTheKeyboard

Copy link
Copy Markdown
Author

The macOS-deps=enabled failure is geotag_test in the bash tests. #9556 fails the same test on the same job (https://github.com/Exiv2/exiv2/actions/runs/41097188073), so I don't think it comes from this change.

To check, I ran the meson suite locally on Fedora 44 with clang 22, using the macOS job's options (-Dauto_features=enabled -Dwarning_level=3 -Dnls=disabled -Db_lundef=false -Db_sanitize=address,undefined) plus -Dcurl=disabled, because there is no libcurl on this machine. I ran it once on main at be479c1 and once on this branch at 51f559e.

The two runs match. geotag_test passes on both. Unit Tests, bugfixTests, lensTests, regressionTests, tiffTests and versionTests pass on both. bashTests fails on both with the same two tests, io_test and stdin_test, where exiv2 exits with -9. Since that happens on main too, it looks local to this machine.

That's Linux only, so it doesn't explain what geotag is doing on the macOS runner.

isXmpType reads up to 80 bytes. On a shorter file it returned false
without seeking back, leaving the stream at EOF, so every detector
registered after XMP (gif, psd, tga, bmp, jp2, the video formats, bmff)
failed too. A 43-byte GIF or a 58-byte BMP was reported as "unknown
image type".

Restore the position on a short read, as isEpsType already does. Do
the same when an empty sidecar (just the XMP packet header) is matched
and advance is false.
@JayOfTheKeyboard
JayOfTheKeyboard force-pushed the fix-short-file-type-detection branch from 51f559e to 103bb8a Compare November 6, 2026 20:17
@JayOfTheKeyboard

Copy link
Copy Markdown
Author

Rebased onto main to pick up d2dad86 ("Avoid sharing the geotag test image with the bugfix suite"). That commit fixes the geotag_test failure on the macOS job. bashTests and bugfixTests run in parallel and both used test/tmp/FurnaceCreekInn.jpg, so the bugfix suite could copy a fresh image back in between the GPS delete and the print. The failed run has 11 GPS lines after the delete step, which fits.

With the macOS job's options on Linux, geotag_test passes on the rebased branch. The workflows are waiting for approval to run again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant