[BUG] Honor data_stat_full for fine-tuning delta-bias statistics #6101

Bug summary

model.data_stat_full=true does not apply to the delta-bias recalibration performed by --finetune. The fine-tuning path uses change-by-statistic, but compute_output_stats disables the full-data scanner whenever model_forward is present and falls back to sampled statistics.

This can leave rare elements without an effective energy-reference correction, even when a shared HDF5 file already contains valid full-dataset absolute output statistics.

Requested interface: reuse the existing model.data_stat_full option for both ordinary output statistics and fine-tuning delta-bias statistics. Do not introduce a separate fine-tuning flag.

DeePMD-kit version

Observed with 3.2.1.dev45+gde3fc8f62.d20261021 on the outisli development branch with local modifications. The same fallback is also present in upstream master at a78bdfe8d5dc494a9d9a397e4a9e103a84205eaa (source links below).

Backend and installation

PyTorch 2.13.0+cu130; editable source installation on Linux. The observed case fine-tunes a spin-free DPA4 checkpoint into a native-spin model.

Input, command, and observed behavior

Relevant configuration excerpt, merged into an otherwise valid fine-tuning input:

{
  "model": {
    "data_stat_full": true,
    "data_stat_nbatch": 10
  },
  "training": {
    "stat_file": "all_399k.hdf5",
    "training_data": {
      "systems": "all_399k.lmdb",
      "batch_size": "mix:2000"
    }
  }
}

dp --pt train input.json --finetune pretrained.pt --skip-neighbor-stat

Startup emits:

Falling back to sampled output statistics: a full scan supports neither delta bias nor stats_distinguish_types=False.

In the observed MagPT case, the target collection contains 399,168 frames and 48 elements. Full absolute statistics were rebuilt successfully: the HDF5 records output_stat_full_scan=true and all 48 observed elements, and a scratch-training checkpoint loads those biases exactly. However, the sampled fine-tuning calibration leaves V and Ag near zero:

That sampled fine-tuning run starts with a training energy RMSE of approximately 159 eV/atom. Enabling data_stat_full on the subsequent fine-tuning launch still produces the fallback warning above.

The absolute and fine-tuning biases are not expected to be identical: fine-tuning must account for the energy already predicted by the pretrained model. The problem is that requesting full-data recalibration still falls back to a sample that does not adequately cover the target elements.

Steps to reproduce

1. Prepare a target dataset with rare elements that are not adequately represented in the default statistics sample, and a compatible pretrained checkpoint with different energy references.
2. Generate valid full-dataset absolute output statistics in an HDF5 cache.
3. Fine-tune the checkpoint on the target dataset with model.data_stat_full=true, pointing training.stat_file at that cache.
4. Observe the fallback warning and inspect the recalibrated element biases rather than assuming that the HDF5 values were reused.

Cause

The fine-tuning initialization explicitly selects change-by-statistic:

• Fine-tuning bias update

The output-statistics implementation disables the full scanner for a model-dependent residual and bypasses the absolute-statistics cache:

• Full-scan fallback and cache bypass

Bypassing an absolute-label cache for model-dependent residual statistics is appropriate. Falling back to sampled data despite data_stat_full=true is the missing behavior.

Expected behavior and acceptance criteria

• Reuse model.data_stat_full: when true, delta-bias recalibration visits every eligible target-training frame and fits the residual between the target labels and the current pretrained model predictions.
• Accumulate residual-regression statistics in bounded-memory batches, retaining the existing type mapping, frame filtering, and bias-adjustment semantics.
• Keep model-dependent residual statistics separate from absolute-label HDF5 statistics. Do not substitute cached absolute biases for the residual regression or overwrite the shared absolute-statistics cache with model-specific values.
• Preserve the existing sampled behavior when data_stat_full=false, and retain the existing absolute-statistics path for training from scratch.
• Add a focused regression test with an element absent from the initial sample but present later in the dataset. Full delta-bias calibration should match an explicit all-frame least-squares reference and produce the appropriate correction for that element.