Quality Engineer, working on trust in AI systems.
Two directions: using agents to run tests β and building the evidence layer that decides whether what an agent did was actually correct.
Shanghai, China Β· [email protected]
My background is in test development: frameworks, automation suites, CI pipelines, turning raw regression runs into signals people actually act on. When agents became capable of taking actions rather than just suggesting them, the interesting question shifted: how do you let an agent execute a test and still trust the result?
That question has two halves, and I work on both. One is using agents to run tests β across five device platforms, interruptible and resumable, with verdicts that trace back to real evidence. The other half is harder and much less crowded: testing the agents themselves. A model doesn't fail the way code fails. It drifts, it confabulates, it passes for the wrong reason β and most of the tooling we have was built for deterministic systems that fail loudly.
I build rather than study: from architecture through implementation, packaging, and release to end users.
- Shipping AI-driven test execution to real users, and fixing the class of bugs that only appear at scale β device drift, context overflow, runs that die halfway through.
- Working out what verification means for non-deterministic systems: behavioral contracts, evals, and what actually counts as proof that an agent did the thing it claims.
- End-to-end product delivery. Architecture, implementation, packaging, release. I've carried a desktop application through the full lifecycle, including the parts that only surface after packaging β native dependencies, update channels, cross-platform breakage.
- Agents reliable enough to hold side effects. Durable execution state, checkpoint and resume, context budgeting against a real model window, and retries that refuse to re-run what they can't prove is safe. In practice: journal every action so an interrupted run picks up without repeating itself, and fence ownership so a stale worker can't write into an execution it no longer owns.
- One instruction layer over heterogeneous devices. Android, iOS, HarmonyOS, Web, Desktop β five transports with different scaling, latency, and failure modes β behind a single abstraction.
- Coverage as a property of the model, not of diligence. Derive cases from a structured model of the product instead of hand-writing them; when a path is ambiguous, escalate to a human rather than letting the agent guess.
- The scarce skill has moved from writing tests to verifying agents. Non-deterministic systems pass for the wrong reason. Verification has to be evidence-based, not assertion-based.
- Requirements outrank generated plans. A generated plan is a derivative artifact; original intent has to stay a first-class contract that gets re-injected on every resume. Otherwise a coarse plan silently redefines the goal, and nobody notices until the report looks wrong.
- Flakiness is a design problem, not a model problem. If a run can't be resumed, its context isn't bounded, and its evidence isn't recorded, no model upgrade will save it.
- Keep humans in decisions that cost something β ambiguity, destructive side effects, money. Automate everything else aggressively.
Deepest in β TypeScript, Python, Java test automation architecture, agent runtimes AI / agents β LangChain, LangGraph, OpenAI-compatible endpoints, Ollama, MCP, context engineering Automation β Midscene, Playwright / Puppeteer, ADB + scrcpy, WebDriverAgent Platforms β Android, iOS, HarmonyOS, Web, Desktop App / UI β React, Vite, Electron, Ant Design, G6 Data β SQLite & MySQL (Kysely), LevelGraph & Neo4j, S3-compatible object storage Delivery β Docker, GitHub Actions, cron scheduling, electron-builder
[email protected] β happy to talk about agent reliability, verification for non-deterministic systems, or multi-platform device automation.
