AI Evaluation & Research Engineering Portfolio

Fu Xiaonan builds runnable evaluation artifacts, agent-reliability tooling, and correctness fixes for AI and ML systems.

The common standard is concrete: isolate a failure mode, add a meaningful control, make the check reproducible, and state exactly what the evidence does and does not establish.

For a concise overview aimed at research-engineering internships or research collaborations, see the one-page pitch.

30-Second Evidence

• Agent reliability: AgentRunProof runs deterministic scenarios against the real OpenAI Agents Runner without a model-provider request, and writes content-addressed evidence for runtime invariants. A maintainer-authored follow-up, openai-agents-python#4414, explicitly references the validation thread on #4413.
• ML systems: merged MLX #4267 fixes incorrect Metal reduction results on negative-stride views by correcting signed pointer arithmetic, with cross-backend regression coverage.
• Open source: 15 authored pull requests merged across 12 external repositories as of August 16, 2026, spanning evaluator correctness, tool-call identity, multimodal data handling, retrieval checks, and deterministic testing.

AgentRunProof is also available as PyPI v0.2.0 and is listed in the externally maintained awesome-auditable-ai collection. These are diagnostic, validation, and curation signals—not OpenAI adoption or endorsement of the library.

Flagship Projects

Project	Question	Public repo	Quick gate
AgentRunProof	Can an OpenAI Agents runtime contract be reproduced and checked without a model-provider request?	Repository	agentrunproof probe basic-tool-session-parity
TraceUseAudit	Does a model's final answer behaviorally depend on supplied traces, or merely imitate trace form?	Repository	make public-check
StateBind Guard	Can coding-agent handoffs preserve executable state bindings rather than only visible identifiers?	Repository	bash scripts/check_public_ready.sh
Claim-Boundary Audit	Which code-review feedback claim is warranted: local utility, transfer utility, text sensitivity, or hold?	Repository	make public-check

Shared Evaluation Pattern

Modern AI evaluation often overclaims from surface success:

• a model can produce a rationale without using it;
• an agent can see an identifier without preserving the executable binding;
• feedback can improve one target-selection surface without supporting broad repair claims.

The artifact pattern is:

1. define the narrow behavioral claim;
2. add counterfactual or control conditions;
3. produce a compact result card;
4. state the claim boundary;
5. make the artifact runnable before making it persuasive.

Evaluation Snapshots

TraceUseAudit

Core message:

high trace accuracy is not evidence of trace use.

Demo	Trace accuracy	Gold final accuracy	Trace-flip	Shortcut-flip
Canonical parity	0.900	1.000	0.100	1.000
Language eligibility	1.000	1.000	1.000	0.333

StateBind Guard

Core message:

visibility is not binding; executable handoffs must preserve active target, semantic role, and exact executable handle.

Method	Accuracy	Unsafe accept rate
visibility baseline	0.500	1.000
keyword-role baseline	0.500	1.000
StateBind Guard	1.000	0.000

Claim-Boundary Audit

Core message:

code-review feedback can support a bounded target-selection claim without proving broad executable repair or human-level review understanding.

Provider	Control	Positive bridges	Row-pooled delta
DeepSeek	shuffled-feedback	4/4	+0.247 [+0.194, +0.300]
DeepSeek	no-feedback	4/4	+0.228 [+0.175, +0.281]
GLM-4.5	shuffled-feedback	4/4	+0.264 [+0.211, +0.317]
GLM-4.5	no-feedback	4/4	+0.244 [+0.197, +0.294]

Direction

I am interested in the space between AI research and research engineering: evaluation, agentic workflows, post-training behavior, ML systems correctness, and tools that make model and agent behavior easier to reason about.