hessboost

Fast, deterministic gradient boosting in Rust (with Python bindings). hessboost provides multi-core tree building with runtime-detected NEON and AVX2 SIMD, strict parameter validation, reproducible models on any thread count, and stable model storage. It supports modern extensions like conformal prediction, explainable boosting machines (EBMs), distributional modeling, and tree-based diffusion, alongside bidirectional XGBoost JSON/UBJSON model interchange.

The name comes from the Hessian: hessboost fits each tree to the loss's gradients and second derivatives (Newton boosting).

Why hessboost

Fast. Multi-core training with runtime-detected NEON and AVX2 kernels; see docs/performance.md.
Deterministic. The same parameters, data, and seed produce the exact same model on any thread count.
Strict. Invalid parameters and unsupported combinations fail loudly; nothing is silently ignored.
Stable storage. Models saved natively are forwards-compatible across releases.
Modern modeling. Built-in support for conformal intervals, Boulevard confidence bands, explainable boosting machines (EBMs), distributional boosting, SGLB uncertainty, tree-based diffusion, in-place updates, and compact models.
XGBoost compatible. Accepts standard XGBoost parameter, objective, and metric names, with bidirectional JSON and UBJSON model interchange.

Getting started

cargo add hessboost

For users pinning a release series in a Cargo manifest:

hessboost = "0.2"

Needs Rust 1.93 or newer and a C compiler (to build libzstd).

use hessboost::prelude::*;

fn main() -> Result<()> {
    // 100 rows × 4 features, row-major, and one label per row.
    let (n_rows, n_cols) = (100, 4);
    let x: Vec<f32> = (0..n_rows * n_cols).map(|i| (i % 17) as f32 / 17.0).collect();
    let y: Vec<f32> = x.chunks(n_cols).map(|row| 2.0 * row[0] - row[1]).collect();

    let dtrain = DMatrix::from_dense(&x, n_rows, n_cols)?.with_labels(&y)?;

    let params = TrainingParams::builder()
        .objective(Objective::SquaredError(RegLoss::default()))
        .tree_method(TreeMethod::Hist)
        .max_depth(6)
        .eta(0.1)
        .subsample(0.9)
        .build()?;

    let model = train(&params, &dtrain, 200)?;
    let preds = model.predict(&dtrain, Iterations::Best)?;
    println!("first prediction: {}", preds.get(0, 0).unwrap());

    model.save("model.bin", ModelFormat::Binary)?;
    let reloaded = BoostedModel::load("model.bin", ModelFormat::Binary)?;
    assert_eq!(reloaded.predict(&dtrain, Iterations::Best)?, preds);
    Ok(())
}

For eval sets, early stopping, custom objectives, or continued training, use Trainer (the xgb.train keyword-argument equivalent):

use std::num::NonZeroUsize;

let result = Trainer::new(&params, &dtrain, 1000)
    .eval(&dvalid, "valid")
    .early_stopping_rounds(NonZeroUsize::new(20).unwrap())
    .train()?;
let model = result.model; // predicts with the best iteration

To ship a self-contained (e.g. static) binary, compile the model file into it; it decodes on first use:

use hessboost::model::EmbeddedModel;

static MODEL: EmbeddedModel = EmbeddedModel::new(include_bytes!("model.bin"), ModelFormat::Binary);

let preds = MODEL.get()?.predict(&dtest, Iterations::Best)?;

Every type and option is in the API docs; runnable programs live in examples/ (cargo run --release --example <name>):

Example	Shows
`train_regression`	end-to-end regression with feature importance
`binary_classification`	a watched eval set, early stopping, AUC
`balanced_bagging`	LightGBM class-stratified sampling for imbalanced binary classification
`multiclass`	per-class probabilities and predicted classes
`ranking` / `rank_xendcg`	LambdaMART and XE-NDCG with query bagging
`constraints`	monotone and interaction constraints, categorical features
`custom_objective`	a custom loss and eval metric
`shap`	SHAP contributions and interaction values
`model_io`	native and XGBoost JSON/UBJSON save and load
`conformal`	calibrated prediction intervals
`boulevard_inference`	confidence intervals for `f(x)` and prediction intervals
`ebm`	an explainable boosting machine's shape functions and their confidence bands
`distributional`	predictive distributions, intervals, and NLL
`virtual_ensembles`	SGLB posterior sampling: knowledge uncertainty rising off the training data
`tree_diffusion`	sampling multimodal and skewed `p(y \| x)` with tree diffusion and flow matching
`forest_flow`	synthetic tabular rows and imputation with ForestFlow / ForestDiffusion
`ordered_target_stats`	encoding a high-cardinality categorical
`compact_model`	reuse penalties and the compact model format
`budget`	budget training against default and tuned training
`online_update`	adding and deleting training rows in place, and exact unlearning
`pfn_boost`	boosting from a pretrained model's logits
`metal`	CPU vs GPU prediction (macOS, `--features metal`)

Python

python/ holds the Python package (pip install hessboost): DMatrix, train, cv, and Booster (taking XGBoost's parameter names), scikit-learn estimators, pandas and polars categorical input, and the conformal, distributional, tree-diffusion, ForestFlow, in-place update, ordered target statistics, budget training, and compact model extras (on macOS, Booster.to_gpu() batch-predicts on the Metal GPU):

import hessboost

booster = hessboost.train(
    {"objective": "binary:logistic", "max_depth": 4}, hessboost.DMatrix(X, label=y), 100
)
probabilities = booster.predict(X_test)

See python/README.md.

Features

Core boosting: gbtree, dart, and gblinear boosters, boosted random forests, and exact, hist, and approx tree methods with native missing-value and categorical support.
Objectives & metrics: Regression (squared, log, Huber, quantile, expectile), binary/multiclass classification, ranking (LambdaMART, XE-NDCG), count, and survival (Cox, AFT), plus typed objective and metric APIs and custom loss hooks.
Validation & workflow: Cross-validation (including purged and forward time-series folds, whole-query ranking folds, and per-fold target statistics), early stopping, feature importance, SHAP values and interactions, model slicing, and iteration ranges.
Interchange: Native binary and JSON formats, XGBoost JSON/UBJSON import/export, LightGBM model import, and models embedded in the binary at compile time.
Modern modeling (opt-in):
- Conformal intervals: Finite-sample coverage guarantees.
- Boulevard inference: Asymptotic confidence and prediction intervals for f(x).
- Explainable boosting machines: Interpretable cyclic GAM/GA²M models with shape-function confidence bands.
- Distributional boosting: Full predictive distributions per row (dist:normal, dist:gamma, etc.).
- Uncertainty & virtual ensembles: SGLB posterior sampling and model shrinkage.
- Generative tabular modeling: Tree-based conditional diffusion, flow matching, and ForestFlow synthetic data and imputation.
- In-place updates: Fast incremental learning and exact or approximate unlearning.
- Compact models: Bit-packed model format with identical margins.
- Budget training: Training controlled by one budget value, based on PerpetualBooster.
- Metal GPU: Apple Silicon GPU prediction and training (--features metal).

Caveats

Full technical details, invariants, and statistical assumptions are documented in the API reference.
Approximate in-place updates are designed for incremental shifts (under ~1% of rows); larger changes benefit from a retrain.
Asymptotic Boulevard inference and prediction intervals require specific noise and structure assumptions; see the inference docs for conditions and empirical coverage validation.

Not implemented

Distributed and external-memory training.
CLI and C bindings.
GPU training outside macOS (a wgpu backend is planned).
A few XGBoost options exist at one setting only, and a few metrics are missing; the API docs list them.

Contributing

AGENTS.md has the build, lint, and test commands and the project's invariants; scripts/README.md covers the XGBoost parity suite and benchmark harnesses.

License and attribution

hessboost is not affiliated with or endorsed by the XGBoost project, and contains no XGBoost source code.

The error function used by the AFT normal distribution is ported from glibc 2.41's s_erf.c, derived from Sun Microsystems' fdlibm (Copyright (C) 1993 Sun Microsystems, Inc.); src/objective/distributional/special.rs carries its notice.

hessboost 0.2.2