[MT] Monotonic constraint. by trivialfis · Pull Request #12341 · dmlc/xgboost

trivialfis · 2026-07-21T22:44:24Z

Use the optimal shared weight when the split doesn't meet the constraint. Instead of rejecting the split, let it continue the evaluation with this shared weight.
Reject reduced gradient.
Ensure categorical features histogram sort doesn't use the constraint to calculate the scores.
Cleanup tests.

Copilot

Pull request overview

This PR extends monotonic-constraint handling to multi-output (vector-leaf) training paths by making split-time monotone bounds target-aware, propagating those bounds across splits, and ensuring reduced-gradient bounds don’t incorrectly constrain full-dimensional leaf refresh. It adds/updates C++/CUDA evaluators, training updaters, tests, and documentation to cover the new behavior.

Changes:

Make TreeEvaluator monotone bounds target-aware and add constrained per-target split-weight computation for vector leaves.
Wire monotone-constraint propagation into CPU/GPU multi-target histogram evaluators/updaters, including reduced-gradient leaf refresh behavior.
Expand unit/integration tests and update docs for vector-leaf monotone constraints and categorical partitioning documentation placement.

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/python/test_monotone_constraints.py	Parametrize parent-gain test across multi-output strategies; tighten type check (`is list`).
tests/python-gpu/test_monotonic_constraints.py	Parametrize GPU parent-gain test across multi-output strategies.
tests/cpp/tree/test_split_evaluator.cc	Add multi-target monotone-bound propagation and zero-hessian constrained-gain coverage.
tests/cpp/tree/hist/test_evaluate_splits.cc	Update evaluator construction/signatures for target-aware evaluators.
tests/cpp/tree/gpu_hist/test_multi_evaluate_splits.cu	Update GPU multi-evaluator initialization/reset API usage in tests.
src/tree/updater_quantile_hist.cc	Pass `n_targets` into `HistMultiEvaluator`; disable constraints for full leaf refresh under reduced gradients; remove vector-leaf monotone restriction.
src/tree/updater_gpu_hist.cuh	Pass `n_targets` into GPU multi-evaluator; apply splits with host+device candidate spans; disable constraints for full leaf refresh; remove vector-leaf monotone restriction.
src/tree/updater_colmaker.cc	Update `TreeEvaluator` construction with explicit `n_targets=1`.
src/tree/split_evaluator.h	Introduce target-aware bounds storage/indexing; implement constrained split weights for vector leaves; add vector `AddSplit` propagation; extend evaluator API with `use_constraint`.
src/tree/hist/evaluate_splits.h	Use constrained split weights for vector leaves; propagate bounds after applying split; add `n_targets` to `HistMultiEvaluator`; use `Evaluator(use_constraint)` API.
src/tree/gpu_hist/multi_evaluate_splits.cuh	Refactor to `unique_ptr<TreeEvaluator>`; make reset accept `n_targets`; add constraint toggle in evaluator access; extend ApplyTreeSplit signature.
src/tree/gpu_hist/multi_evaluate_splits.cu	Allocate `TreeEvaluator` with `n_targets`; enforce per-target constrained split weights; propagate bounds via `AddSplit` on applied splits; tighten runtime checks.
src/tree/gpu_hist/leaf_sum.cu	Ensure rounding/target dimension consistency; use target-aware `CalcWeight`.
src/tree/gpu_hist/evaluator.cu	Update `TreeEvaluator` construction to pass `n_targets=1`.
src/tree/gpu_hist/evaluate_splits.cuh	Update `GPUHistEvaluator` `TreeEvaluator` construction to pass `n_targets=1`.
python-package/xgboost/testing/multi_target.py	Add reduced-gradient monotone constraint test ensuring leaf refresh isn’t constrained by reduced-coordinate bounds.
python-package/xgboost/testing/monotone_constraints.py	Extend parent-gain test helper to handle multi-output strategy and expected gains.
doc/tutorials/multioutput.rst	Remove vector-leaf categorical partitioning section (relocated).
doc/tutorials/monotonic.rst	Document vector-leaf monotone constraint behavior and reduced-gradient caveat.
doc/tutorials/categorical.rst	Add vector-leaf categorical partitioning documentation (moved from multioutput tutorial).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 20 out of 20 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

src/tree/gpu_hist/multi_evaluate_splits.cuh:109

MultiHistEvaluator now stores tree_evaluator_ as a unique_ptr, but GetEvaluator() dereferences it without a guard. If a caller accidentally uses the evaluator before calling Reset(), this will be a null dereference at runtime; adding an explicit CHECK(tree_evaluator_) makes the contract clear and fails fast.

  [[nodiscard]] auto GetEvaluator(bool use_constraint) const {
    return tree_evaluator_->GetEvaluator<GPUTrainingParam>(use_constraint);
  }

trivialfis · 2026-07-22T00:51:09Z


 .. versionadded:: 2.0.0

-.. note::


there's a similar note at the beginning of the doc.

trivialfis · 2026-07-22T12:37:26Z

Extracted a function for the pooled weight; reused the existing weight calculation function by sharing a train param, renamed from the existing GPU Training param.
Added a unit test to compare the results.
Dropped the workaround for value gradient.

trivialfis added 8 commits July 22, 2026 03:40

[wip][MT] Monotonic constraint.

4054c29

reduce grad.

3c05acf

doc.

9dcdc4c

GPU.

46643e8

lint.

a138ca0

cleanup.

8f32f2e

Cleanups.

fbcc6c7

doc.

a472b90

trivialfis requested a review from Copilot July 21, 2026 22:44

trivialfis mentioned this pull request Jul 21, 2026

[Roadmap] Multiple outputs. #9043

Open

48 tasks

Copilot started reviewing on behalf of trivialfis July 21, 2026 22:46 View session

doc.

b4c2d4a

Copilot AI reviewed Jul 21, 2026

View reviewed changes

Comment thread src/tree/split_evaluator.h

Comment thread src/tree/gpu_hist/multi_evaluate_splits.cuh Outdated

trivialfis added 3 commits July 22, 2026 07:31

tests.

5e94e20

cleanups.

06d8494

cleanups.

40f4507

trivialfis requested a review from Copilot July 21, 2026 23:40

Copilot started reviewing on behalf of trivialfis July 21, 2026 23:40 View session

Copilot AI reviewed Jul 21, 2026

View reviewed changes

Comment thread python-package/xgboost/testing/monotone_constraints.py

trivialfis added 2 commits July 22, 2026 07:44

Cleanup.

40fb68f

rng.

ef99554

trivialfis changed the title ~~[wip][MT] Monotonic constraint.~~ [MT] Monotonic constraint. Jul 21, 2026

trivialfis marked this pull request as ready for review July 21, 2026 23:47

trivialfis requested a review from RAMitchell July 21, 2026 23:48

trivialfis added 3 commits July 22, 2026 07:56

Cleanups.

c3ad7b4

Promote the vector leaf from WIP.

d503a7d

lint.

fdd6681

trivialfis commented Jul 22, 2026

View reviewed changes

trivialfis added 2 commits July 22, 2026 08:59

Cleanups.

0344237

checks.

f80327e

RAMitchell approved these changes Jul 22, 2026

View reviewed changes

Comment thread src/tree/gpu_hist/multi_evaluate_splits.cu

Comment thread src/tree/split_evaluator.h Outdated

Reuse weight fn, add test.

750f932

Note.

579b356

trivialfis merged commit 7a9210b into dmlc:master Jul 22, 2026
82 checks passed

trivialfis deleted the mt-mono-constraint branch July 22, 2026 21:01

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

[MT] Monotonic constraint.#12341

[MT] Monotonic constraint.#12341
trivialfis merged 21 commits into
dmlc:masterfrom
trivialfis:mt-mono-constraint

trivialfis commented Jul 21, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

trivialfis Jul 22, 2026

Uh oh!

Uh oh!

Uh oh!

trivialfis commented Jul 22, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Uh oh!

Conversation

trivialfis commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

trivialfis Jul 22, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

trivialfis commented Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

trivialfis commented Jul 21, 2026 •

edited

Loading

trivialfis commented Jul 22, 2026 •

edited

Loading