Skip to content

Latest commit

 

History

History
97 lines (68 loc) · 9.64 KB

paper.md

File metadata and controls

97 lines (68 loc) · 9.64 KB
title ART: A Modular Transformer Library for Uncertainty-Aware Language Reasoning
tags
Python
PyTorch
deep learning
natural language processing
uncertainty quantification
mixture-of-experts
low-precision arithmetic
authors
name affiliation orcid
Swadhin Biswas
1
0009-0005-2980-6651
name affiliation
Tuhin Sheikh
1
name affiliation
Md Shahriar Shakil
1
affiliations
name index
Department of Computer Science and Engineering, Daffodil International University, Dhaka, Bangladesh
1
date 23 August 2026
bibliography paper.bib

Summary

art-transformer (AURORA-Transformer, ART) [@biswas2026art] is an open-source Python library, built on PyTorch [@paszke2019pytorch], for building language models that know when they do not know. Instead of one dense transformer doing everything at once [@vaswani2017attention], ART splits a reasoning pass into six inspectable stages (perception, reasoning, routing, execution, verification, synthesis) and threads an explicit uncertainty signal through all of them. That signal decides which sub-tasks run, whether an expert's answer may be trusted, and, when the evidence runs out, whether the model should simply refuse to answer. The library ships the full training pipeline, a benchmark harness covering six standard evaluation suites, YAML configurations for single-model and ensemble deployments, a FastAPI inference server, and a pytest suite covering every stage.

Statement of need

Large language models deliver wrong answers in exactly the same confident tone as right ones. In domains such as medicine, law, and finance, a calibrated refusal is worth far more than a fluent guess, yet the techniques that address this problem live in separate corners of the literature and of our codebases: calibration methods [@guo2017calibration; @naeini2015obtaining], selective prediction [@elyaniv2010foundations; @geifman2017selective], deep ensembles [@lakshminarayanan2017simple], mixture-of-experts routing [@jacobs1991adaptive; @shazeer2017outrageously; @fedus2022switch], retrieval grounding [@lewis2020retrieval], and tool use [@schick2023toolformer]. Each is usually studied in isolation and buried inside monolithic model implementations.

Researchers who want to experiment with routing policies, verification gates, abstention thresholds, or low-precision (4-bit) training dynamics therefore lack a modular, hackable testbed. ART was built to fill that gap. Every forward pass emits uncertainty as an explicit output alongside its predictions, each stage can be swapped out or ablated on its own, and the whole stack was designed from the outset for native 4-bit (NVFP4) computation on Blackwell-class GPUs [@nvidia2025nvfp4; @micikevicius2018mixed], so numerical-noise robustness can be studied alongside reliability. It is intended for machine-learning researchers studying modular architectures and selective prediction, and for practitioners who need an inference stack that can decline rather than hallucinate. We use it in our own research, and every artefact reported in the repository was produced with it.

The story of one forward pass

A dense model reads a question and starts writing an answer immediately; understanding, planning, computing, and checking are tangled into a single stream of hidden states. ART tells a different story: it walks the question through six stations, and nothing reaches the user until the last station signs off.

ART's six-stage pipeline. The uncertainty channel U emitted by the Cognitive Transformer Core feeds both the Dynamic DAG Router and the Context & Reality Anchor, gating what executes and what gets accepted.

The journey begins at the Binary Perception Layer (art/modules/bpl.py). Tokenisation happens offline: text is pre-compiled into fixed-width 32-bit records that are memory-mapped at training time, so the runtime loop reads contiguous integer arrays instead of parsing strings. This removes the CPU-bound data bottleneck that throttles modern accelerators, at the cost of some vocabulary flexibility relative to byte-level [@xiao2024efficient] or unigram [@kudo2018sentencepiece] schemes, a trade we discuss in the repository documentation.

Next, the Cognitive Transformer Core (art/modules/ctc.py) reasons over those embeddings. It contains no decoder and no vocabulary softmax. Its final states are projected into three disjoint channels: reasoning representations R, control signals C, and a sigmoid-bounded uncertainty channel U, trained jointly with a lightweight auxiliary correctness objective. This is where ART departs most sharply from convention. Confidence is not read off a generation softmax (known to miscalibrate under distribution shift and quantization [@hendrycks2017baseline; @jiang2021confidence]); it is learned as a dedicated feature.

Control passes to the Dynamic DAG Router (art/modules/d3r.py). Five control primitives (EXECUTE, COMPOSE, BRANCH, AGGREGATE, VERIFY) are selected by deterministic argmax against projection vectors frozen after training, so routing carries no sampling variance. The router assembles acyclic task graphs with hard caps on depth and width, and silently suppresses any sub-task whose uncertainty exceeds a configurable threshold. Because acyclicity is enforced on insertion, the recursive runaway loops familiar from agent frameworks cannot happen here.

Each surviving node is dispatched through an ExpertRegistry (art/modules/experts/) to stateless domain experts (mathematics, code, retrieval, passthrough). Experts share no memory and cannot call one another; they return structured results only, so cross-talk between sub-tasks is impossible by construction. The math expert routes parseable expressions through SymPy for exact symbolic evaluation and falls back to a neural stand-in otherwise.

Before any result counts, the Context & Reality Anchor (art/modules/cra.py) checks it against R and U and returns ACCEPT, REJECT, or ABSTAIN. Finally, the Synthesis Head (art/modules/syn.py) aggregates only accepted results via confidence-weighted attention over verified candidates, producing the final answer or an explicit abstention when too little survives verification.

For higher stakes, Multi-ART runs M independent instances over the same input and merges their verified outputs via uncertainty-weighted voting, extending deep-ensemble ideas [@lakshminarayanan2017simple] without shared weights or joint training. Disagreement triggers abstention rather than forced consensus.

Uncertainty as an explicit output

Every call returns an ARTOutput dataclass containing the answer, an explicit abstained flag, calibrated confidence, pooled uncertainty, and diagnostics about the routed graph:

import torch
from art import ART

model = ART(vocab_size=50257, hidden_dim=768,
            num_layers=12, num_heads=12)
out = model(input_ids=torch.randint(0, 50257, (2, 128)))

print(out.abstained)    # True when evidence was insufficient
print(out.confidence)   # calibrated confidence estimate
print(out.num_tasks)    # sub-tasks executed after uncertainty gating

Downstream systems can act on refusals programmatically (escalate to a reviewer, retry with retrieval, log for audit) instead of parsing prose for hesitation that isn't there.

What ships in the box

Beyond the six modules above, the repository provides everything needed to train, evaluate, and serve models end to end:

  • Training pipeline (art/training/) driven by YAML configurations (configs/art_base.yaml, configs/art_full.yaml, configs/multi_art.yaml), including staged schedules that pre-train the core before freezing routing projections.
  • Benchmark harness (art/benchmark/) covering GSM8K [@cobbe2021gsm8k], MATH [@hendrycks2021math], MMLU [@hendrycks2021mmlu], HumanEval [@chen2021humaneval], MedQA [@jin2021medqa], and LegalBench [@guha2023legalbench], plus a failure-injection framework and latency measurement; results are written as structured JSON so figures regenerate exactly.
  • NVFP4 utilities (art/utils/quantization.py) implementing block-scaled 4-bit simulation with bounded-initialisation safeguards, alongside optional hardware-accelerated paths for Blackwell GPUs.
  • Serving (art/api/serve.py): a FastAPI application exposing /v1/infer, /v1/batch, and health endpoints.
  • Model interoperability: checkpoint export integrates with Hugging Face tooling [@wolf2020transformers].

Evaluating a trained model under simulated NVFP4 with ensemble consensus takes one command:

uv run python -m art.benchmark.run \
    --checkpoint checkpoints/best.pt \
    --nvfp4 --multi --limit 500 --output-dir result

Quality control

Every stage of the pipeline has unit tests (tests/test_modules.py) covering shape contracts, cycle rejection, depth/width bounds, uncertainty gating, verification outcomes, and abstention logic; the suite runs on CPU in seconds. The benchmark harness writes per-table JSON files consumed by the analysis module, so every number and figure in our reports traces back to a reproducible script rather than a notebook cell.

Honest limitations

ART is a research testbed, not a production product. The bundled experts are intentionally lightweight neural stand-ins behind a stable interface (only the math expert dispatches to a real symbolic backend today), and users bring their own heavier tools. Native NVFP4 acceleration requires Blackwell-class hardware; elsewhere the format is faithfully simulated. And because the architecture refuses to guess, it abstains more often than dense baselines: useful in a hospital, occasionally annoying in a chatbot. We consider this trade-off the point.