Skip to content

Latest commit

 

History

History
47 lines (27 loc) · 6.04 KB

data-to-models.md

File metadata and controls

47 lines (27 loc) · 6.04 KB

From training data to model releases

I wanted the data work behind my GLM experiments to be useful beyond one checkpoint. That meant keeping code-training examples, domain-specific adapter data, and expert-pruning calibration distinct, then building an import path that could carry selected material into the Hy3 project.

The result is a public dataset, source-tracked import code, a shared verification interface, and published model experiments. This case follows those artifacts through the pipeline. It reviews existing records; it does not claim a new training run or a fresh verification of the whole corpus.

What I published

GLM-5.2 Demolition training and calibration data contains 154 JSONL files at the inspected revision:

Material Files Role
heal/ 150 Code examples, agent tool-use conversations, repair data, and domain-specific training material
calib/ 4 Prompts and mixed corpora used to measure expert saliency before pruning

The default Dataset Viewer indexed 87,586 rows on September 10, 2026: 84,231 training, 3,277 validation, and 78 test rows. Its schema exposes messages, facet, and elite_signature. These are the viewer's default splits, not a count of every example in every repository file. The dataset card reports 272,549 examples across the broader collection; that larger total was not independently recounted for this case study. Neither figure establishes a count of unique examples.

Making the data usable in another model project

The Hy3 import script makes several choices explicit:

  1. Sample with limits. Each source has a domain label and a sampling cap. Reservoir sampling uses a fixed seed, so a large source does not automatically dominate the selected material.
  2. Normalize the training schema. Chat roles and tool calls are converted into the message representation the training pipeline consumes. Domain and source-path metadata travel with the example.
  3. Separate training from calibration. Supervised examples retain assistant responses. Calibration takes prompt text through a separate path and writes a separate output file.
  4. Write an import receipt. Counts, skipped rows, source distribution, domain distribution, and split sizes are saved alongside the normalized outputs.

The recorded import reports 338 imported SFT examples, 1,536 calibration prompts, and nine duplicate SFT rows removed. After combination with seed data, it records 313 training, 17 validation, and 17 test examples. This import includes additional repair inputs beyond the public dataset; it is not a claim that the public release alone recreates the entire training mixture.

Deduplication also has a precise scope: the script removes identical normalized message sequences within the imported SFT material. It then combines those rows with seed data and re-splits. That is not a guarantee against duplicates across the full combined corpus or against benchmark contamination.

Checking generated material and learning from repairs

The GLM project's verification layer dispatches to language toolchains, SQL execution, Lean, and design checks. Results carry a stage, diagnostics, and output so callers can distinguish what was checked.

The trace harvester takes tasks marked solved after repair and turns their final solutions into supervised examples. Unsolved tasks go into a separate file with their harness and last diagnostic. It consumes recorded results; it does not independently rerun each solution while harvesting.

Verification depends on the check performed. In this version, missing toolchains and unsupported languages can return passed=True with stage="skip". A caller must inspect the stage; the Boolean alone does not establish successful execution. I therefore describe the tools and their scope rather than claiming every published row was compiled, tested, or independently validated.

Connecting the pipeline to model decisions

The Hy3 REAP25 plan report records pruning from 192 to 144 experts across 79 MoE layers, protecting all 11 domain facets. Its saliency checks retained an average 87% of score mass, with no layer below 70%. Those are pruning-plan checks, not model-quality scores.

The separate paired evaluation records 45/46 cases for REAP25 against 46/46 for Lite and returns REVIEW, with a token-cap truncation identified. The model card also documents a tool-call separator regression that the suite did not cover.

This is the work behind a model release: designing the data flow, preserving source information, making the selection process inspectable, and measuring the resulting behavior separately from whether the pipeline ran successfully.

All case studies · Data and model catalog