Module leaf_model 

Available on crate feature alloc only.

Pluggable leaf prediction models for streaming decision trees.

By default, leaves use a closed-form weight computed from accumulated gradient and hessian sums. Trainable leaf models replace this with learnable functions that capture more complex patterns within each leaf’s partition.

Leaf model variants

Model	Prediction	Overhead	Best for
ClosedFormLeaf	constant weight -G/(H+lambda)	zero	general use (default)
LinearLeafModel	w . x + b (AdaGrad-optimized)	O(d) per update	low-depth trees (2–4)
MLPLeafModel	single-hidden-layer neural net	O(d*h) per update	complex local patterns
AdaptiveLeafModel	starts constant, auto-promotes	shadow model cost	automatic complexity allocation

AdaGrad optimization

LinearLeafModel uses per-weight AdaGrad accumulators for adaptive learning rates. Features at different scales converge at their natural rates without manual tuning. Combined with Newton scaling from the hessian, this gives second-order-informed, per-feature adaptive optimization.

Exponential forgetting

LinearLeafModel and MLPLeafModel support an optional decay parameter that applies exponential weight decay before each update. This gives the model a finite memory horizon, adapting to concept drift in non-stationary streams. Typical values: 0.999 (slow drift) to 0.99 (fast drift).

Warm-starting on split

When a leaf splits, child leaves can inherit the parent’s learned function via LeafModel::clone_warm. Linear children start with the parent’s weights (resetting optimizer state), converging faster than starting from scratch.

Adaptive promotion

AdaptiveLeafModel runs a shadow model alongside the default closed-form model. Both are trained on every sample, and their per-sample losses are compared using the second-order Taylor approximation. When the Hoeffding bound (the tree’s existing delta parameter) confirms the shadow model is statistically superior, the leaf promotes – no arbitrary thresholds.

Structs

AdaptiveLeafModel

Leaf model that starts as closed-form and promotes to a more complex model when the Hoeffding bound confirms it is statistically superior.

ClosedFormLeaf

Leaf model that computes the optimal weight in closed form: weight = -grad_sum / (hess_sum + lambda).

LinearLeafModel

Online ridge regression leaf model with AdaGrad optimization.

MLPLeafModel

Single hidden layer MLP leaf model with ReLU activation.

Enums

LeafModelType

Describes which leaf model architecture to use.

Traits

LeafModel

A trainable prediction model that lives inside a decision tree leaf.