sft-loop

Public MIT loop for same-seed before/after PEFT supervised fine-tuning on one pinned open instruct model at or under 3B parameters.

The scored train path is TRL + PEFT LoRA SFT on Cap M1 Pro 32GB (MPS). It does not use Unsloth. Unsloth stays an optional CUDA-only extra and is not required for weight_change_claimed.

The checked-in Phase 3 result in this checkout is fail-closed unsupported. This environment has no MPS device and does not have the TRL + PEFT train extra installed, so no train step ran and no before/after task scores exist. Scored runs/ artifacts are not invented here. Cap runs the train on the Mac after this path lands and checks in the scored report plus proof.

Pinned stack

Component	Pin	Role
TRL	trl==0.24.0	Scored-path trainer (trl.SFTTrainer)
PEFT	peft>=0.8.0	LoRA on the MPS scored path (TRL 0.24 extra floor)
Transformers	transformers>=4.56.1	Model load on the MPS scored path
Model	Qwen/Qwen2.5-1.5B-Instruct	Pinned instruct model
Method	PEFT LoRA supervised fine-tuning (sft-lora)	Rank 8, 2 optimizer steps
Device	mps on Cap M1 Pro 32GB	Report field when that path scores
Unsloth	unsloth==2026.9.12	Optional CUDA-only. Not used on MPS. Not required for weight_change_claimed.

Qwen/Qwen2.5-1.5B-Instruct is a public Hugging Face instruction-tuned model with 1.54B parameters under Apache-2.0. 1.54B is inside the ≤3B bound.

trl==0.24.0 requires transformers>=4.56.1, datasets>=3.0.0, and accelerate>=1.4.0. Those floors are the train extra in pyproject.toml, together with torch>=2.2 and peft>=0.8.0. The MPS path loads the model in float32 with eager attention. It does not use 4-bit quantization.

trl==0.24.0 is the highest release inside the range Unsloth 2026.9.12 declares: trl>=0.18.2,!=0.19.0,<=0.24.0. That pin is the optional CUDA package. It is not installed for the MPS scored path.

Honesty

• Soft-PASS is unused and is refused. It is not a status and not a gate outcome.
• This repository is not an AdaptiveSandbox qualification. Reports have no adaptiveSandboxQualified field.
• score ≠ safety. A task score is a task metric. The report field metric_role is fixed to task_score_not_safety.
• This is not frontier RL.
• This is not full pretraining.
• No invented uplift: the checked-in fail-closed Phase 3 report has no before, no after, and no task_score. weight_change_claimed is false. The only scored JSON in this repo is Cap's MPS run at runs/phase3/scored.json. CPU verification does not check in a scored report.
• weight_change_claimed can be true only on a scored report whose train_proof matches a checked-in LoRA runs/phase3/proof/adapter_config.json and a TRL SFTTrainer log runs/phase3/proof/train_log.txt with global_step >= 1. Unsloth is not required for that claim. On MPS the report labels device mps, backend trl-peft, and stack.unsloth is unused.

Task

Synthetic held-out task uppercase-word-v1. The grader is case-sensitive first-token exact match (token == item_id.upper()). Train words and eval words are disjoint. The metric is a task score, not a safety judgment.

Quantization

Comparison of CPU fp32, CPU dynamic int8, ONNX int8, T4 fp32, T4 fp16, and T4 NF4 on the pinned model. The table is generated by python -m sft_loop.quant table. Do not edit the numbers by hand. NF4 is paired only with fp32 on the same T4. T4 fp16 is not a pairing baseline. GPU rows are not paired with CPU rows. ONNX is not a pairing. T4 kernel v1 is withdrawn. Kernel v4 (inaveed101/sft-loop-quant-nf4) is the measured GPU source. ONNX kernel v4 (inaveed101/sft-loop-quant-onnx-int8) is measured collapse: exact match 0/129, case-folded n/a because this arm has no raw generations. Teacher-forced target NLL is 7.25 versus CPU fp32 0.80 and CPU int8 10.03, so the zero score is the same class of dynamic-int8 collapse, not a generate-loop-only bug. Peak RSS was 28.99 GB (86% of the 33.66 GB host) at quantize. scored p50 ~5.7 s is eight no-cache forwards of 1.5B on 4 CPUs. T4 fp16 is unsupported (fp16 overflow on T4, 0/129), not a measured arm. The NF4 79/129 exact-match cell is case-sensitive first-token match. Case-folded match is 126/129 on both NF4 and t4-fp32. Sequence NLL rose under NF4. See docs/DECISIONS.md.

Fixture quant-heldout-v1 sha256 f0f813c96d56dc346c7cb6705e6927275fdef03f44c4cffb67bb70b3af8668d8 revision 989aa7980e4cf806f80c7fef2b1adb7bc71aa306 seed 0 items 129 threads 1 warmup 2 repeats 5 scored decode greedy max_new_tokens 8 throughput decode min_new_tokens=8 max_new_tokens=8.

precision	status	on-disk bytes	peak rss bytes	peak gpu bytes	scored p50 ms	scored p95 ms	mean generated tokens	decode tokens/s (min=max=8)	exact match	case-folded match	n	accelerator	limiter
fp32	measured	3098955668	7058575360	n/a	1996.4266	2690.6479	3.248062	1.9207317	0.36434109	n/a	129	n/a	rss
int8	measured	2477926394	15464464384	n/a	1193.9633	1310.3139	7.9457364	6.4966602	0.0	n/a	129	n/a	rss
onnx-int8	measured	1779618939	28993339392	n/a	5701.7247	8204.2175	7.9379845	1.3662138	0.0	n/a	129	n/a	rss
t4-fp32	measured	3098955668	5299331072	6238088192	144.00548	181.8304	3.248062	25.466484	0.36434109	0.97674419	129	NvidiaTeslaT4	gpu
fp16	unsupported	unsupported	unsupported	unsupported	unsupported	unsupported	unsupported	unsupported	unsupported	unsupported	unsupported	unsupported	probed_failure
nf4	measured	1143625737	5398913024	1224451584	227.77117	291.58627	3.4108527	15.383386	0.6124031	0.97674419	129	NvidiaTeslaT4	gpu
nf4-fp16	unsupported	unsupported	unsupported	unsupported	unsupported	unsupported	unsupported	unsupported	unsupported	unsupported	unsupported	unsupported	probed_failure

comparison	status	mean item delta	standard error	bootstrap low	bootstrap high	n
int8 minus fp32	paired	-0.36434109	0.042536442	-0.4496124	-0.27906977	129
nf4 minus t4-fp32	paired	0.24806202	0.041224898	0.17054264	0.3255814	129
nf4-fp16 minus t4-fp32	unavailable	unavailable	unavailable	unavailable	unavailable	nf4-fp16_unmeasured

case-folded comparison	status	mean item delta	standard error	bootstrap low	bootstrap high	n
int8 minus fp32	paired	n/a	n/a	n/a	n/a	129
nf4 minus t4-fp32	paired	0.0	0.01556432	-0.031007752	0.031007752	129
nf4-fp16 minus t4-fp32	unavailable	unavailable	unavailable	unavailable	unavailable	nf4-fp16_unmeasured

Paired rows share one hardware class. NF4 is paired only with fp32 on the same T4. int8 is paired only with CPU fp32. GPU versus CPU is not paired. T4 fp16 is not a pairing baseline. scored p50/p95 allow EOS. decode tokens/s is a separate fixed-length pass. Exact match is case-sensitive first-token equality with item_id.upper(). Case-folded match is first-token equality after str.casefold(). n/a means the arm has no raw generations to rescore.

runs/ path

Canonical schema: schema/run_report.schema.json (JSON Schema draft 2020-12).

Every object sets additionalProperties to false. Status is unsupported, unscored, or scored.

Path	status	Contents
runs/fixtures/unsupported.json	unsupported	Scaffold fixture for the blocked shape. No before or after.
runs/fixtures/unscored.json	unscored	Placeholder with no scores and no weight-change claim.
runs/phase3/unsupported.json	unsupported	Live Phase 3 loop result for seed 0 on a machine with no MPS and no CUDA Unsloth stack. Same seed was requested for both eval splits. Neither split ran.
runs/phase3/scored.json	scored	Cap M1 seed 0, Qwen/Qwen2.5-1.5B-Instruct, TRL + PEFT, device mps. Held-out exact match is 0.375 before and 0.375 after (n 8). weight_change_claimed is true because the proof files match.
runs/phase3/proof/adapter_config.json	proof	PEFT LoRA config from that run (r 8). Adapter weights stay gitignored.
runs/phase3/proof/train_log.txt	proof	TRL SFTTrainer log: global_step 2, train_loss 4.7694361, device=mps, backend=trl-peft.

.gitignore ignores scratch files under runs/ and un-ignores the fixture directory, runs/phase3/unsupported.json, runs/phase3/scored.json, the two proof filenames, the quant projections, and runs/quant/kernel/. Adapter weights (*.safetensors and the adapters/ directory) stay ignored.

Fail-closed unsupported path

When the requested device or the train dependencies are unavailable, each command below prints one JSON report whose status is unsupported. The report does not set device, so it does not claim MPS or CUDA. Soft-PASS is unused and is refused.

python -m sft_loop
python -m sft_loop.loop --model-id Qwen/Qwen2.5-1.5B-Instruct --seed 0
python -m sft_loop.loop --model-id Qwen/Qwen2.5-1.5B-Instruct --seed 0 --device mps
python -m sft_loop.train --model-id Qwen/Qwen2.5-1.5B-Instruct --seed 0
python -m sft_loop.eval --model-id Qwen/Qwen2.5-1.5B-Instruct --seed 0 --split before
python -m sft_loop.eval --model-id Qwen/Qwen2.5-1.5B-Instruct --seed 0 --split after

sft_loop.train and sft_loop.eval do not print a task score on this path. weight_change_claimed is false. before and after are absent. python -m sft_loop.loop without --device is the command that wrote runs/phase3/unsupported.json.

Condition	unsupported.reason
MPS is unavailable, or torch / transformers / PEFT / TRL / datasets / accelerate is not usable, and the optional CUDA Unsloth stack is also unusable	gpu_or_train_deps_unavailable
The requested device is mps and MPS or the TRL + PEFT libraries are not usable. Missing Unsloth does not by itself block this path	gpu_or_train_deps_unavailable
The requested device is cuda and CUDA or Unsloth or TRL is not usable	gpu_or_train_deps_unavailable
The pinned stack is usable and this invocation did not finish a train step	train_not_run

Both reasons are fail-closed. unsupported is not a pass.

To copy the probe JSON onto a scratch path (gitignored, outside the tracked carve-outs):

python -m sft_loop --out runs/scratch/unsupported.json

Reproduce on this checkout (no MPS)

Commands that ran for the checked-in unsupported artifact:

python -m pip install -e ".[dev]"
pytest
python -m sft_loop
python -m sft_loop.loop --model-id Qwen/Qwen2.5-1.5B-Instruct --seed 0 --out runs/phase3/unsupported.json
python -m sft_loop.train --model-id Qwen/Qwen2.5-1.5B-Instruct --seed 0
python -m sft_loop.eval --model-id Qwen/Qwen2.5-1.5B-Instruct --seed 0 --split before
python -m sft_loop.eval --model-id Qwen/Qwen2.5-1.5B-Instruct --seed 0 --split after

Cap M1 Pro 32GB (MPS) scored path

Cap runs this on the Mac. It is TRL + PEFT LoRA SFT without Unsloth. Do not install the unsloth extra for this path. The loop evals the held-out words before training and again after training with the same --seed. A finished step writes proof under runs/phase3/proof/ (adapter_config.json and train_log.txt with global_step >= 1). The scored report is stdout and, with --out, runs/phase3/scored.json. Adapter weights land in adapters/phase3/ and stay gitignored.

python -m pip install -e ".[dev,train]"
python -m sft_loop.loop \
  --model-id Qwen/Qwen2.5-1.5B-Instruct \
  --seed 0 \
  --device mps \
  --run-id phase3-mps-seed0 \
  --out runs/phase3/scored.json

Check the report before committing it:

• status is scored
• device is mps
• backend is trl-peft
• stack.unsloth is unused
• weight_change_claimed is true only together with the two proof files
• before.seed and after.seed equal seed
• metric_role is task_score_not_safety

If the command cannot finish a real step, it still prints unsupported, omits device, and does not fill in before/after numbers.

Files Cap checks in after that run:

• runs/phase3/scored.json
• runs/phase3/proof/adapter_config.json
• runs/phase3/proof/train_log.txt

Optional Unsloth (CUDA-only)

Unsloth is not the MPS scored path and is not required for weight_change_claimed. On a CUDA machine the loop can instead run a 2-step Unsloth + TRL LoRA SFT. The report then labels device cuda and backend unsloth. This was not run for the checked-in unsupported report.

python -m pip install -e ".[dev,train,unsloth]"
python -m sft_loop.loop --model-id Qwen/Qwen2.5-1.5B-Instruct --seed 0 --device cuda

Phase 4 (Kaggle GPU, two-phase)

Phase 4 is a sibling of the uppercase fixture, not a new sft_loop task. The held-out task is tenk-extract-v1: open-book figure extraction from capitalcite passages at commit 34b42ee0b387738725705a0af682dd3f030a9944. That pin is exact. Eval n is 123. The grader is canonical-figure exact match. runs/phase4/scored.json exists. It is assembled from Kaggle kernel v3 at ee7f5c8511d42916c02147b68731e6c0a36cd317 (tarball sha256 421df4ada47a4b8d98c22c8c0f606cc6eef038760bc62dfb15124ae4768cf507) and regraded from the committed raw generations.

slice	exact_match	n	source
before	0.6585365853658537 (81/123)	123	runs/phase4/raw/eval_before.json, reassembled after the $(370)$ fix
after	0.7317073170731707 (90/123)	123	runs/phase4/raw/eval_after.json
cluster delta (primary)	0.07317073170731707, 95% CI [0.0, 0.11214953271028037]	123 rows, 10 filings	runs/phase4/scored.json cluster_ci, 10000 rounds, seed 0
item delta (secondary)	0.07317073170731707, 95% CI [0.0, 0.14634146341463414]	123	runs/phase4/scored.json paired_ci, 10000 rounds, seed 0

One T4 run. Same eval ids, split hash, seed 0, greedy decode, hardware. Baseline NLL 0.6888449788093567 is finite and is the first eval row only. 102 distinct before predictions. No ! collapse. Trained in fp32. fp16.json is unsupported (fp16_amp_overflow: AMP loss non-finite, first overflow layers.0.self_attn.o_proj.base_layer). Adapter sha256 e94fb237398a3e496552014b5ddb8dad6556a2ac56a669d480bbf6098f5e0109, 8745704 B, gitignored. Contamination status is same-issuer overlap (JPM 2025 CIK 0001628280, 6 eval rows kept). Format gains 5, substantive gains 10, losses 6, net new correct figures +4.

What ran

Kernel inaveed101/phase4-tenk-lora v3 on 2x Tesla T4, 5087 s wall. Dataset unchanged since ce57285. Raw outputs are in runs/phase4/raw/. Adapter weights are gitignored.

Accelerator: 2x Tesla T4, 15,636,037,632 B each, compute capability 7.5. bf16 is unsupported. DDP is unsupported. fp16 AMP is unsupported (see fp16.json). Train dtype is fp32, global_step 200. Checkpointing off peaked at 13,905,077,760 B. On peaked at 8,269,284,864 B. Both fit the T4. Profiler peak 8,416,858,112 B. aten::mm was 76.23% of self CUDA time.

Interview notes

LoRA stores a low-rank update ΔW = BA. Rank r is 8. Alpha is 16, so the applied scale is alpha/r = 2. Target modules are the attention projections q_proj, k_proj, v_proj, o_proj: those matrices route which year-column figure to copy, and they fit a 15 GB T4 with sequence length 1024. IEEE fp16 max is 65504. GradScaler only protects small fp16 gradients and requires fp32 master weights; it does not fix an Inf activation in RMSNorm, attention, or lm_head. Run 2 loaded the 1.5B weights as float16, wrote a 0/123 !!!!!!!!!!!!!!!! baseline, then died on a non-finite loss at step 1. bf16 has a wider exponent and does not need that scaler, but T4 cannot run it. Gradient checkpointing trades extra forward compute for lower activation memory; the kernel measures both peaks at the same batch and sequence length. Effective batch is 8 (microbatch 1 times 8 accumulation steps). The LR schedule is cosine with 5% warmup so the first steps do not jump from a cold adapter. Eval n=8 cannot show uplift: SE of a Bernoulli rate is sqrt(p(1-p)/n) ≈ 0.18 at p=0.5, which swallows a typical LoRA delta. This task uses n >= 100 and a contamination check (exact and 5-gram Jaccard near-duplicates between train and eval, eval figures against the static prompt template, and issuer-family overlap). The locked split has no shared issuer string. It still has JPM family overlap via CIK 0001628280.

The filing-cluster interval covers 0. Verdict is no uplift. The item-level mean is positive. The honest line is inconclusive / no uplift at the filing level, with a positive item-level estimate. Train loss interpolated by step 7. Eval ran only at step 200. That is an overfitting risk. This is one kernel run. Do not retune it.

License

MIT. See LICENSE.