Background

Pointer probing is a feature of 3D since PR #118. This allows validating
structures that contain pointers by checking that those pointers are valid, and
then copying the pointed-to contents into a destination buffer, and validating
the copied bytes.

This PR extends probing to support validating structures that may contain either
32-bit or 64-bit pointers, depending on the architecture for which the sending
process is compiled. Structures that contains such pointers may have different
layouts depending on the architecture, due to differences in alignment padding
etc.

We want a smooth way to specify that a 3D description of a data structure should
handle multiple pointer widths, enabling a single specification that can be
specialized to either 32 or 64-bit layouts. Additionally, we want to be able to
easily copy a 32 bit structure to 64-bit layout, so that the downstream code can
proceed uniformly with the two cases.

Three new features

Pointer type with explicit representations

This PR adds a new qualifier to pointer types, to give them an explicit
representation, as shown by the following example:

aligned
typedef struct _B(EVERPARSE_COPY_BUFFER_T dst) {
  UINT32 b1;
  A *pointer(UINT64) //new qualifier
     pa 
     probe
     ProbeCallback(length=sizeof(A), destination=dst)
}

Only the following unsigned integer type can be a legal pointer qualifier:
• pointer(UINT32)
• pointer(UINT64)

The current default behavior is retained, i.e., an unqualified pointer type
defaults to UINT64.

Fine-grained probing (for traversing 32-bit pointers)

We want to allow probing and reading structures with 32-bit pointers into 64-bit
kernel data structures. This involves copying pieces of 32-bit structures
carefully, adding padding as needed, coercing pointers from 32 to 64-bit
representations.

For this, this PR introduces a new sub-language for fine-grained probing of
pointers, as the following example illustrates.

Let’s start with a simple structure A: nothing fancy here, just two UINT32 fields.

aligned
typedef struct _A {
	UINT32 a1;
	UINT32 a2;
} A;

Next, we have a structure B, which contains a pointer to an A. The
representation of A is the same on both 32-bit and 64-bit architectures. So, we
can just probe and copy the referenced A structure in a single shot to an out
parameter, a_out.

extern probe ProbeAndCopy

aligned
typedef struct _B (EVERPARSE_COPY_BUFFER_T a_out) {
	UINT32 b1;
	A *pa probe
	   ProbeAndCopy(length=sizeof(A), destination=a_out);
	UINT32 b2;
	UINT32 b3;
} B;

Next, we have a structure C, which contains a 32-bit pointer to a B. When
following a 32-bit user mode pointer, we assume that the data pointed to is in a
32-bit layout, including all the pointers it contains.

As such, when following the pointer pb, B is in 32-bit representation, different
from the 64-bit representation expected by the kernel. So, we probe the pointer
pb repeatedly, and copy pieces of B over to the out parameter b_out.

aligned
typedef struct _C (EVERPARSE_COPY_BUFFER_T a_out, EVERPARSE_COPY_BUFFER_T b_out)
{
	UINT32 c1;
	B(a_out) *pb probe pointer(UINT32)
	   (destination=b_out) {
		//copy first 4 bytes for b1
ProbeAndCopy (length=4); 
		//add 4 bytes of padding
      constant (length=4; 0x00uy);
		//read 4 bytes, coerce to pointer using UlongToPtr, and write 8 bytes
           UlongToPtr(ProbeAndReadU32); 
//read 8 bytes for b2, b3
		ProbeAndCopy (length=8); 
   }
} C;

Note, instead of a single call to ProbeAndCopy, we have a block with several
operations writing into the destination buffer b_out. The result is the b_out
contains representation of the 32-bit data converted to 64 bit layout. After the
copy is completed into b_out, we run the validator B(a_out) on the copied data,
ensuring that the data satisfies the expected 64-bit layout, as well as all
refinement constraints etc. that the type B may have.

Auto-specialization

Of course, explicitly writing fine-grained probes is very tedious and
error-prone. Also, we would like to define the data format compactly once, for a
64-bit layout, and obtain the equivalent 32-bit layout almost for free.

So, we support the following automated specialization feature:

extern probe ProbeAndCopy
extern probe (WITH_OFFSETS) ProbeAndCopyLen
extern probe (READ UINT32) ProbeAndReadU32
extern probe (WRITE UINT32) WriteU32
extern probe (WRITE UINT64) WriteU64
extern PURE UINT64 UlongToPtr(UINT32 ptr)

aligned
typedef struct _A {
	UINT32 a1;
	UINT32 a2;
} A;

aligned
typedef struct _B (EVERPARSE_COPY_BUFFER_T a_out) {
	UINT32 b1;
	A *pa probe
	   ProbeAndCopy(length=sizeof(A), destination=a_out);
	UINT32 b2;
	UINT32 b3;
} B;

aligned
typedef struct _C64 (EVERPARSE_COPY_BUFFER_T a_out, EVERPARSE_COPY_BUFFER_T b_out)
{
	UINT32 c1;
	B(a_out) *pb probe
	  ProbeAndCopy(length=sizeof(B), destination=b_out);
} C64;

specialize (pointer(UINT64), pointer(UINT32)) C64 C32;

The semantics of the specialize directive is to specialize the given type C64
to use the given pointer size (pointer(UINT32)) and name the specialized type
C32. This elaborates to the explicit fine-grained probing definition shown
earlier.

Now, one can define a uniform entry point function that multiplexes the two
definitions, based on the requestor mode.

entrypoint
casetype _C (
Bool IsRequestor32,
EVERPARSE_COPY_BUFFER_T a_out,
EVERPARSE_COPY_BUFFER_T b_out)
{
    switch(IsRequestor32) {
	  case true:
		C32(a_out, b_out) c32;
      case false:
		C64(a_out, b_out) c64;
    }
} C;

Generated code

The example above produces the following code:

The header file contains signatures for the extern declarations.

#ifndef __Specialize3_ExternalAPI_H
#define __Specialize3_ExternalAPI_H

#if defined(__cplusplus)
extern "C" {
#endif

#include "EverParse.h"

extern BOOLEAN ProbeAndCopy(uint64_t uu___, uint64_t x0, EVERPARSE_COPY_BUFFER_T x1);

extern BOOLEAN
ProbeAndCopyLen(
  uint64_t uu___,
  uint64_t x0,
  uint64_t x1,
  uint64_t x2,
  EVERPARSE_COPY_BUFFER_T x3
);

extern uint32_t ProbeAndReadU32(BOOLEAN *uu___, uint64_t x0, uint64_t x1);

extern BOOLEAN WriteU32(uint32_t uu___, uint64_t x0, EVERPARSE_COPY_BUFFER_T x1);

extern BOOLEAN WriteU64(uint64_t uu___, uint64_t x0, EVERPARSE_COPY_BUFFER_T x1);

extern uint64_t UlongToPtr(uint32_t ptr);

#if defined(__cplusplus)
}
#endif

#define __Specialize3_ExternalAPI_H_DEFINED
#endif

Internally, the generated validators are produced in a higher-order style, passing probe functions as arguments, to copy or coerce-and-copy the pointed-to data, though the external signature hides this altogether.

uint64_t
Specialize3ValidateC(
  BOOLEAN IsRequestor32,
  EVERPARSE_COPY_BUFFER_T AOut,
  EVERPARSE_COPY_BUFFER_T BOut,
  uint8_t *Ctxt,
  void
  (*ErrorHandlerFn)(
    EVERPARSE_STRING x0,
    EVERPARSE_STRING x1,
    EVERPARSE_STRING x2,
    uint64_t x3,
    uint8_t *x4,
    uint8_t *x5,
    uint64_t x6
  ),
  uint8_t *Input,
  uint64_t InputLen,
  uint64_t StartPosition
);